Dream of the Multiversal Cylinder

(I usually reserve this blog for speculations on intellectual topics, but last night I had a dream that seemed sufficiently interesting to post here. So, here goes ;-) ....

In this dream, I moved to a strange foreign nation, and met a beautiful girl there whose ex-boyfriend was making her life very difficult, yet who she was still somehow attached to....

His martial arts expertise alarmed me, and so together with the mother of a friend who lived in this same strange place -- a very short, hunchbacked old lady who walked with a cane and wore a funny straw hat -- I went to a weird old-fashioned section of the city, where we did two things.

First, we paid some old white-bearded "witch doctor" to cast a magical spell on the ex-boyfriend, which caused him to forget having ever known the girl, haha.

Then, we went to a strange store full of ancient relics, and bought this cylindrical wooden container, which I was supposed to keep in my bedroom for good luck, but not to open.

The girl and I walked along the beach and the ex-boyfriend walked right past and showed no sign of recognizing her. This freaked her out a bit, and she asked me to have the spell undone on Dec. 21 2012.

Then I went back to my house, which I suddenly shared with the girl, and of course I had to open the wooden cylinder. She kept telling me not to, but I had to anyway. I opened one end of it, prying it open with a screwdriver, and inside the small cylinder was an infinite space -- a whole multiverse of possibilities.

She just kept staring inside it, looking intent but not saying anything. I asked if she wanted me to close it; but she shook her head no. There were millions of these little intelligent creatures in there, which could see our (and everything's) past and future.... Clearly she was absorbing a lot of knowledge from them ... and so was I ... but it was also clear that we were absorbing somewhat different things.

Then, we looked at each other and, without words, asked each other if we should dive into one of those universes or stay in this one. It was clear that in those universes we could still exist as individuals (and could still be with each other); but would exist in radically different form (some form not constrained by time, though there were other constraints not comprehensible in human terms).

Gradually, we collectively realized that we did not feel like entering that other multiverse at that particular time.

Then, she gave me a look that meant something like: "I will never be afraid of anything relating to human society anymore, nor be afraid of my own emotions, because I can see that this whole world of you and me and humanity and Earth is just a sort of artistic construction, which exists for aesthetic purposes. We have chosen to remain in this universe so as to remain part of this artwork ... "

... and then her unspoken thought faded out before it was done, because someone was in the house walking around and we got distracted by wondering who it was...

... and then I woke up because of the noise of my dad walking around downstairs in my house (he was visiting last night)

... and I tried to fall back asleep so as to re-enter the dream, but failed ...

Causation as Storytelling

I've pointed out before (and it's not my original observation) that no branch of modern science contains a notion of "cause" more than vaguely similar to the folk psychology notion -- causation, as we commonsensically understand it, is something that we humans introduce to help us understand the world; and most directly, to help us figure out what to do....

[David Orban, in a comment on an earlier version of this post, noted that some formulations of relativity theory contain the term and concept of "causation." But causation as used in that context is really just "influence" -- the restrictions on light cones and so forth tell you which events can influence which other ones, but don't tell you how to distinguish which events are causal of which other ones in the stronger, commonsense usage of the "causation" concept.]

Cause, in our everyday intuitive world-view, is tied to will: "A causes B" means "I analogically intuit that if I were able to choose to make A happen, then B would happen as a result of my choice."

And, I think cause is also tied to storytelling. Causal ascription is basically a form of storytelling.

Think about the archetypal story structure:

If we envision a typical causal ascription as fitting into this structure, we have:

The Setup is the situation in which the causation occurs, the set of "enabling conditions." For instance, we rarely would say that oxygen is the cause of us being alive -- oxygen is considered an "enabling condition" rather than a cause of our life ... it's part of the set-up....

The Confrontation is the introduction of something unusual into the Setup. This must be something that is not always there in the Setup, otherwise one wouldn't be able to isolate it as the cause of some particular events. It's not necessarily a violent confrontation, but it's a violation of the norm. Could be someone shooting a gun, could be a couple having sex, could be a finger pushing down on a computer keyboard. The less expectable and frequent it is, the better -- i.e. the more convincing it will be as a potential cause of some event.

The End is the event being caused.

My suggestion is that, if one digs into the matter deeply, one will find that many of the same patterns distinguishing compelling stories from bad ones, also distinguish convincing causal ascriptions from unconvincing ones.

What Would Aristotle Say in a Situation Like This?

The Aristotelian distinction between efficient and final cause is also relevant here.

"Efficient cause" is what we usually think about as causation these days: roughly, A causes B if

P(B|A,Setup) > P(B|Setup)

and there is some "plausible causal mechanism" (i.e. some convincing story) connecting A and B in the context of the Setup.

"Final cause" is telos, teleology -- A causes B if B, as a goal, somehow reaches back in time and makes A happen as part of the inevitable movement toward B's occurrence.

Modern physics theories have no place for final causes in the Aristotelian sense. But, human psychology does! Very often, when a human seeks a cause for something, what they're doing starting with some event they've observed and trying to find a "good reason" for it.

Why did I fall in love with her? It must have been because she was beautiful ... or smart ... or rich ... or whatever.



Why did my business succeed? It must have been because I was smart ... or because it was the right time ... etc.

Storytelling generally mixes up efficient and final causation in complex ways. Many stories give a feeling of inevitability -- final causation -- by the end. And when postmodernist stories avoid giving this feeling, it's generally done intentionally, with a view toward violating the known psychological norm and doing something disconcerting or shocking.

Convincing causal ascriptions, like compelling stories, tend to mix up efficient and final causation.


Cause and Will

Nietzsche wrote that (paraphrasing) "free will is like the army commander who takes responsibility, after the fact, for the actions of his troops."

Experiments by Gazzaniga, Libet and other neuroscientists have validated that in many cases the reflective, willing portion of the human brain-mind "decides" to do something only well after some other part of the brain has actually already started to do it.

This fits in fine with the notion of causal ascription as storytelling. Willing is a matter of making up a story about how one came to do something. It had better be a compelling story or the illusion of free will will fall apart, which is bad for the maintenance of the self-model!


Causation and Storytelling in Neuroscience, AGI and Early Cognitive Development


One concrete hypothesis that comes out of this train of thought is that, when the neural foundations of causal ascription and storytelling are unravelled, it will turn out that the two share a large number of structural and dynamical mechanisms.

Another hypothesis is that, if we want our AGI systems to be able to ascribe causes in humanlike ways, we should teach our AGI systems to tell and understand stories in a humanlike way.

I strongly suspect that one of the major roles that storytelling plays in human childhood, is to teach children patterns of narrative structure that they will use throughout their lives in constructing causal ascriptions (along with many other kinds of stories).

...

Ahh ... I would love to improve this blog post with a bunch of concrete examples but that will need to wait for later ... I'm tired and need to wake up early in the AM ... at least "I" think that is the cause of me not wanting to improve it right now ;-D ...

AGI, Ethics, Cognitive Synergy and Ethical Synergy (from my Yale talk...)

Earlier this year I gave a talk at Yale University titled "Ethical Issues Related to Advanced Artificial General Intelligence (A Few Small Worries)" ...

It was a verbal discussion focused rather than PPT focused talk but I did show a few slides (though I mostly ignored them during the talk): anyway the brief ugly slideshow is here for your amusement...

The most innovative point made during the talk was a connection between the multiple types of memory and multiple types of ethical knowledge and understanding.

I showed this diagram of different types of memory and the cognitive processes associated with them (click the picture to see a bigger, more legible version)



and then I showed this diagram


which associates different types of ethical intuition with different types of memory.

To wit:

• Episodic memory corresponds to the process of ethically assessing a situation based on similar prior situations
• Sensorimotor memory corresponds to "mirror neuron" type ethics, where you feel another person's feelings via mirroring their physiological emotional responses and actions
• Declarative memory corresponds to rational ethical judgment
• Procedural memory corresponds to "ethical habit" ... learning by imitation and reinforcement to do what is right, even when the reasons aren't well articulated or understood
• Attentional memory corresponds to the existings of appropriate patterns guiding one to pay adequate attention to ethical considerations at appropriate times
  

I presented the concept that an ethically mature person should balance all these kinds of ethics.

This notion ties in with a paper that Stephan Bugaj and I delivered at AGI-08, called Stages of Ethical Development in Artificial General Intelligence Systems. In this paper we discussed, among other topics, Kohlberg's theory of logical ethical judgment and Gilligan's theory of empathic ethical judgment. In the present terms, I'd say Kohlberg's theory is declarative-memory focused whereas Gilligan's theory is focused on episodic and sensorimotor memory. We concluded there that to pass to the "mature" stage of ethical development, a deep and rich integration of the logical and empathic approaches to ethics is required.

The present ideas suggest a modification to this idea: to pass to the mature stage of ethical development, a deep and rich integration of the ethical approaches associated with the five main types of memory systems is required.

Why Bother to Vote?: A Novel Multiversal Answer

Any sensible person with the choice of going to the polls to vote has almost surely asked themselves “Why should I bother voting when it’s incredibly unlikely my vote will make any difference, given the large number of people voting?”

[Note for non-US readers: in the US, unlike some countries, there is no legal requirement to vote; it's an option.]

I've discussed this issue with dozens of people and have never really heard any sensible answers.

I say "If I stay home and work or play Parcheesi instead of voting, then the election will proceed exactly the same way as if I had voted. The odds of me affecting this election are incredibly tiny."

They say: "Yeah, but if EVERYBODY thought that way, then democracy couldn't work." ... as if this were a counterargument.

Or: "Yeah, but if everyone intelligent enough to have that train of thought followed it and avoided voting, then only stupid people would vote and we'd have a government elected by the retarded.... Oh, wait ... would that be any different than what we actually have now?"

I've thought about this a lot, off and on, over the years, and finally I think I've come up with an interesting, novel answer to the question. To have a handy label, I'll call it the "multiversal answer."

This is a somewhat philosophically complex answer, which requires a deviation from our ordinary ways of thinking about our relationship between ourselves and the universe.

I'll run through some details and probability calculations, and then get back to philosophy and free will and such at the end.


Rational Agents

The multiversal answer pertains to agents who make choices based on expected utility maximization. That is, it pertains to agents who: Given a choice between two actions, will choose the one with the property that, after the choice is made, the agent’s utility will be highest. Or, to put it informally, it pertains to agents who follow the rule: “Choose the option that, in hindsight, you will wish you had made.”

Of course people don't always follow this sort of rule in determining their actions; people are complex dynamical systems and don't follow any simple rules. But, my point is to argue why voting might make sense for an agent following a simple rational decision-making procedure. I.e.: why voting might be a reasonable behavior even though, in the sense indicated above, the odds of your vote being decisive in an election are minimal.


Vote So That You'll Live in a Universe Where People Like You Vote

The conceptual basis of the multiversal answer is the principle that “you should vote because, if you vote, this means that after you’ve voted, you’ll know that you probably live in a universe where people similar to you vote.

On the other hand, if you don’t vote, this means you probably live in a universe where people similar to you don’t vote.”

Clearly, you would rather live in a universe where people similar to you vote.

(Yes, this could be formalized based on the degrees to which individuals with varying degrees of similarity to you vote. But we won’t worry about the math details for now.)

Your vote may not count much on its own, but it’s a bad thing if everyone similar to you (with the same preferences as you) doesn’t vote.

Note that it’s not good enough to intend to vote but then back out at the last minute. After all, if you do that, then probably everyone similar to you is going to do the same thing! So if you do that, it means you’re in a universe where people similar to you are likely to almost vote, rather than a universe where people similar to you are likely to vote.

Possible Worlds

Underlying this answer is a “possible worlds” philosophy, holding that there are many possible universes we could live in -- and we don’t know exactly which one we do live in, based on the limited data at our disposal.

So, given a predicate P like “the degree to which people similar to me vote,” we can estimate the truth value of P by a weighted average of the product

(degree that P holds in possible world W) * (probability that world W is the one I live in)

(or some similar formula).

Some Plausible Assumptions

So, based on the above, suppose we we assume that


P(people like me vote | I vote) > P(people like me vote | I don’t vote)

and


P(good world | people like me vote) > P(good world | people like me don't vote)

(where “good world” is shorthand for “I live in a possible world I like.” Again, this can be more fully formalized, but I won’t bother with that for now.)

Note that these probabilities are calculated across possible worlds. For instance,

P(people like me vote | I vote)

means

P(people like me vote in possible world W | I vote in possible world W)

A Critical Question

So, given the above, one has the question: From the above inequalities, can we derive that


P(good world | I vote) > P(good world | I don't vote)

which would imply that voting increases the probability of living in a good world (whether or not one wins the particular election one is voting in)???

The answer is: not quite.

But, sort of.

What the mathematics tells us is that this conclusion holds if

max[ P(gw|Iv & plmv) - P(gw|plmv), P(gw|Iv & ~plmv) - P(gw|~plmv) ]

<

.5 * [P(gw | plmv) - P(gw)] [ P(plmv|Iv) - P(plmv)]


where

gw = good world
Iv = I vote
plmv = people like me vote

Basically, if the left hand side of this equation is small, this means that the effect of me voting on the probability that I live in a good world, is almost entirely contained in the effect of people like me voting on this probability. But, this seems quite sensible.

So, if this condition holds, then voting increases the odds of being in a good world, so it makes some sense to vote to increase the odds of being in a good world.

There’s still a quantitative calculation to make, though. Voting has some cost, so one needs to estimate whether the increase in the expected goodness of {the world one estimates oneself to live in}, induced by voting, outweighs the cost of voting. This devolves into a bunch of algebra that I don’t feel like doing right now. But note that it’s a totally different calculation than the calculation as to whether one’s individual vote makes any difference in a particular election.

Free Will

Underlying the above perspective is an attitude toward "free will" which is different from the one conventional in the modern Western mindset.

In the conventional interpretation of "free will", a person can choose whether to vote or not, and this doesn't impact their estimate of what kind of universe they live in -- it's an independent, free choice.

In the interpretation used in the multiversal answer to the voting problem, a person can (in a sense) choose what to do, but then when they study their choices in hindsight, they can infer from the pattern of their choices something about the universe they live in.

Combining this with the "expectation maximization" approach, which says you should make the choices that you'll be happiest with in hindsight (after the choice is made) ... one comes up with the principle that you should make the choice that, in hindsight, will yield the most desirable implications about what kind of universe you live in.

And it's according to this principle that, in the multiversal answer, voting may be a sensible choice regardless of the small chance that your particular vote impacts the election.

The point is that, after voting, the fact that you voted will give you evidence that you live in a nice universe, where people like you vote and therefore things tend to go in a favorable way for you.

On the other hand, if you don't vote, then afterwards the fact that you didn't vote will give you evidence that you live in a universe where people like you don't vote, and therefore things tend to go against you.

So, I think the decision to not vote because your vote is very unlikely to impact the election, is based partly on a naive folk theory of "free will." In a more mature view of will and its relation to the universe, the decision to vote or not isn't exactly a "free and independent decision" ... but there is rationality in making the "not quite free or independent decision" to vote.

(Perhaps this is related to the intuition people have when they say things like "If everyone thought that way, then no one would vote." Statements like this may reflect some intuition about what it means to live in a good branch of the multiverse, which however conflicts with modern Western folk psychology intuition about free will.)

Will China Build AGI First?

One of the reasons I spent 4 weeks in China this summer, organizing the First AGI Summer School and collaborating on research with my friend Hugo de Garis's Artificial Brain Lab at Xiamen University, was to qualitatively investigate Hugo's stories about the great potential for AGI R&D allegedly existent in the country of China.

One thing I learned about China is: the answer to almost any nontrivial question is some complex, multidimensional form of "maybe" or "sort of."

(Eventually the maybes and sort-ofs must collapse: Chinese do make hiring and firing decisions, get married, publish papers, and take other definitive actions ... but on the whole I found that in China there is a much greater willingness to embrace uncertainty than in the US, and a much smaller desire to make things clear and definite.)

In this spirit, I can't claim I came to a definite conclusion about the potential of China to lead the world in advanced AGI R&D.

But I can say that it's a definite possibility ;-) ....

Though the situation is complex, my gut feel is that Hugo is probably right, at least in the following sense: If a moderate number of AGI researchers (from the West and China both) apply their energy to pursuing the R&D opportunities that China offers, there is a strong potential that AGI research could advance there much faster than in other parts of the world.

Here are some relevant facts, militating in favor of China's role in AGI technology:

• Even in the current dismal world economy, China's economy is still growing
• Chinese students and researchers are willing to work long hours (of course some Americans and Europeans are too ... but my impression is that this willingness is greater in China)
• The Chinese education system is very good at teaching advanced mathematics and algorithm theory, which are important for AGI
• China is interested in pursuing advanced technologies -- both with practical applications in mind, and with a nationalistic motive of displaying their technological strength relative to other nations
• Unlike the US, the Chinese research funding establishment has no "chip on its shoulder" about AI or AGI -- it has the same status as any other advanced technology. There was never an "AI winter" in China, nor is there dramatically more skepticism about AI than about other computer technologies
• Due to the centralized system of government, if the central administration decides they value a certain technology, there is the potential for a massive amount of resources to be directed to that technology in a relatively rapid time-frame. Things often move very slowly in China, but sometimes they can also move much faster than in less centrally organized economies.
• The cost of highly educated labor is low in China, so that if funding from outside China is found to help support China-based projects, this funding can go a long way!

Here are some other relevant facts, which are challenges China has to overcome if it wants to lead the world in AGI:

• The Chinese university education system is narrowly disciplinary, whereas AGI requires copious interdisciplinary knowledge
• Because China lacks a large, robust, cutting-edge software industry, there isn't that much "cultural knowledge" about how to manage complex software projects using "agile" methodologies. Yet, AGI is a complex software project that really demands an agile methodology. (Note that China has a load of great software engineers; the issue I'm pointing out regards software project management, not software engineering.)
• Due to the nature of Chinese culture, it is fairly common for work environments to arise in which participants don't feel free to share their innovative ideas, and to point out problems with the ideas others are pursuing (especially if these "others" have higher social status according to Chinese tradition). However, some Chinese work environments are very friendly to innovation and criticism; so IMO this is best considered as a problem that can be overcome with attention to the personalities involved and the management mechanisms
• Due to various factors including the restrictions on travel and Internet sites that the Chinese government places, China often feels "cut off" from the rest of the world, which results in a less-than-optimal degree of interaction with the international research community

Given the above factors, my conclusion is that IF China is able to attract sufficient foreign AGI experts, it may well be able to leapfrog ahead of other nations in the race to create AGI.

What the foreign experts would bring is not just AGI expertise and ideas, but expertise in allied areas like

• interdisciplinary education
• agile software project management
• the management of innovation-friendly, "flat-hierarchy" research groups

Further, the presence of foreign experts in China full-time would result in other foreign experts more frequently traveling to China to speak, and in Chinese students more often traveling outside China for conferences and research visitation -- all of which would decrease the "China isolation" factor, and increase the intellectual potency of Chinese AGI research labs.

It's also the case that in Chinese academia right now, foreigners can get away with "shaking things up" a little more than Chinese nationals can. So even if a Chinese national showed up with exactly the same expertise and personality as a foreigner, they would have different strengths and weaknesses in the context. They would be able to get some things done more easily due to their Chinese-ness, but would also not be able to "get away with" as many disruptive methodologies.

So: the reason I think foreign AGI experts are critical is not that Chinese lack good ideas about AGI. (Yes, I think my own ideas about AGI are the best ones, but that's not my point right now!) It's that I don't think any one country has a monopoly on great AGI ideas and people, so the prize is likely to go to a country that can build an international AGI research community ... and also that there are certain organizational skills that are very useful for AGI, but not that well developed in China right now.

But there are serious challenges involved in recruiting foreign AGI experts to China:

• The salaries are low by international standards. The salaries foreign faculty get at Chinese universities allow a very nice lifestyle in China -- but even so, they don't go that far in terms of international travel, purchase of electronics, or helping family members in the West.
• Many AGI researchers have spouses and children who don't want to live in China. There are good international schools for researchers' children; but finding appropriate jobs for spouses can be difficult due to the language barrier and the different nature of the economy
• The Chinese system of government is alien and off-putting to some foreigners. (As an example, this blog cannot be read by most Chinese Internet users right now, because blogger.com is blocked by the Chinese government. This sort of thing really bothers some foreign researchers and/or their families)
• Some researchers will fear "career damage" if they go to a university without international name recognition (though, this factor would disappear quickly after a critical mass of researchers went to China)

So, the sixty-four trillion dollar question is to what extent these latter factors can be overcome.

I believe they could be overcome if Chinese universities or research agencies made very clear, very clearly research-friendly offers to foreign AGI researchers -- and then followed through on these offers once the researchers arrived. AGI researchers are a dedicated bunch, and many would put up with the problems cited above in order to have a good chance to lead a team of brilliant, qualified students at implementing powerful AGI systems.

As you may have guessed, one reason I'm cataloguing these factors so systematically is that I'm debating trying to rearrange my life to either move to China or (more likely) spend one semester per year in China.

At the moment, in my own discussions with Chinese universities about AGI research funding, I am finding things mildly confusing. The discussions are going interestingly, but I feel much less clarity than I would in a comparable discussion with a US university. This is nobody's fault -- it's a natural consequence of "cultural differences" -- but it's a factor that will have to be smoothed-out, IMO, if China is going to recruit a sizeable number of foreign AGI researchers.

So we now reach the conclusion of the above chain of thought. IF Chinese universities manage to fine-tune the art of recruiting foreign AGI researchers, then I think that China has a real chance of leading the world in the development of AGI.

I predict that, if China doesn't adopt the world lead in AGI, it will be because it fails at the things I cited above. They will fail to dominate in AGI if it turns out that the Chinese way of recruiting and retaining foreigners is too alien, causing a failure to accumulate a critical mass of foreign R&D leaders. This could certainly happen. Time will tell.

What Are the Risks if China Pulls Ahead in the AGI Race?




So China might plausibly take the lead in the AGI race.

As a citizen of the USA and Brazil (not China), does this worry me?

Not really.

Like Buckminster Fuller, I consider myself a "passenger on Spaceship Earth" (and I won't hesitate much to board another vessel when one becomes available -- or better yet I'd like to send multiple copies of myself on multiple vessels! But, I digress ;-).

One thing I'm being insistent on in my collaboration with Hugo de Garis's Artificial Brain Lab at Xiamen University, is that all our work be released as open-source code. The university folks there have no problem with this. So, in the case of my and Hugo's work, it's not a situation where we're trying to develop an AGI that will be exclusively owned by the Chinese government.

And, I feel strongly that anyone else doing AGI work in China -- or anywhere else! -- should take the same approach. The main reason I decided to open-source my own AGI project (OpenCog) (while keeping some valuable AGI-related technologies proprietary within Novamente LLC), is the intuition that AGI is a sufficiently big and thoroughgoingly important thing that it should be developed by the human collective mind as a whole, not by a small group or even a single nation.

Of course, if more and more AGI research gets done in China, then more and more of the world's AGI expertise will exist in China -- which will give China a substantial leadership position in AGI, regardless of whether the AGI code is open-source. But this really doesn't worry me much either, partly because I've been so impressed with the character and spirit of the younger generation of Chinese, who (in these scenarios we're discussing) will be doing most of the AGI work.

I met wonderful Chinese people of all ages during my visit to China. But there are huge generational differences among Chinese, and the Chinese who grew up with the Internet have a drastically different view of the world than the immediately prior generations. Most of the Chinese I met aged under 30 had a reasonably modern, international understanding of the world -- and some of the Chinese I met aged under 22 had such modern attitudes that they really could have been youth anywhere. The Internet is spreading international ideas and culture around the world, just like it's spreading the AGI meme around the world -- and may increasingly start spreading AGI researchers around the world ... we'll see.

Although I used a picture of Alfred E. Neuman above (like Hugo, I'm kind of a sucker for dramatic effect), I want to be clear that I don't have a cavalier attitude about the threat that could be posed if ANY government took control of the world's first AGI for their own parochial ends.

But I think this is a problem we need to work around, regardless of which country we do our research in.

By developing open-source code (made available on SourceForge, Launchpad, Google Code and so forth), and by carrying out our research in a way that emphasizes linkages with the international research community, we'll guarantee that AGI comes about as a product of the international collective mind of AGI researchers. This provides no grand guarantee of "AGI safety" (nothing can do that), but I strongly feel it's the best approach.

The First AGI Summer School in Xiamen, China (Retrospective and Random Related Musings)

I returned home 2 weeks ago from the First AGI Summer School, which was held in the Artificial Brain Lab at Xiamen University in Xiamen, China at the end of June and the beginning of July.

Ever since I got back I've been meaning to write a proper summary of the summer school -- how it went, what we learned, and so forth -- but I haven't found the time, and it doesn't look like I'm going to; so, this blog post will have to suffice, for the time being at any rate.

First of all, I need to express my gratitude to Hugo de Garis



and Xiamen University for helping set up the summer school. Coming to Xiamen to do the Summer School was a great experience for me and the others involved -- so, thanks much!

Some photos I took in Xiamen are here (mixed up with a few that YKY took on the same trip). (Viewer beware: some of these are summer school photos, some are just "Ben's Xiamen tourism photos"....)

To get a sense of what was taught at the summer school -- and who was on the faculty -- you can go to the summer school website; I won't repeat that information here.

The first two weeks of the summer school were lecture-based, and the last week was a practical, hands-on workshop focused on the OpenCog AI system. Unfortunately I missed most of the hands-on segment, as I wound up spending much of that week meeting with various Chinese university officials about future possibilities for Chinese AGI funding (but I'll write another blog post about that), and demo-ing the Artificial Brain Lab robot to said officials.

See here for some videos of the above-mentioned robot, along with some "OpenCog virtual pet" demo videos that were shown at the summer school. (And, the OpenCog virtual pet was also gotten up and running "live" in Xiamen, of course....)

The number of students wasn't as large as we'd hoped -- but on the plus side, we did have a group of VERY GOOD students who learned a lot about AGI, which was after all the point.


(In fact, most conferences have found their attendance figures down this year, due to people wanting to save money on travel costs: an obvious consequence of the faltering world economy.) The majority of students were Chinese from Xiamen University and other universities in Fujian province, but there were also some overseas students from Europe, the US, Korea and Hong Kong (OK, well, Hong Kong isn't quite "overseas" ;-).

All the lectures were videotaped by Raj Dye (thanks Raj!!)


and will be put online once Raj gets time to edit them. I think these will form an extremely valuable resource, and will reach a lot more people than the summer school itself did. (Long live the Internet!!). Raj's active camera work captured a bunch of the dialogues during and after the talks as well, and I think these will make quite interesting viewing. As you might expect, there was some pretty intense give-and-take (especially, for example, during Allan Combs' talks on cognition and the brain).

I'm definitely interested to help organize some future AGI summer schools ... though the next one will be in a different location, as we've already done a pretty good job of spreading the word about AGI to the AI geeks of Fujian Province! Maybe the next one will even be back here in the boring old US of A ....

Random Observations on Chinese-ness in the AGI Context

I learned a lot about China in the course of doing the summer school (though I'm still pathetically ignorant about the place of course ... there's a lot to know) ... I won't try to convey 1% of what I learned here, but will just write down a few hasty and random semi-relevant observations.

First, I learned to speak verrrry slowly and clearly since Chinese students are more accustomed to written than spoken English! ;-)

More interestingly, I learned that the Chinese educational system is more narrowly disciplinary than the US system, and also more focused on memorization of declarative knowledge rather than practical "know how." Compared to their US counterparts, computer science graduates in China know an AWFUL LOT of computer science, yet don't have much advanced knowledge of areas beyond computer science, nor all that much software engineering knowledge or hands-on coding experience. ("Software Engineering" is a separate department in the Chinese university system, and I didn't get to know the Software Engineering students, only the Computer Science ones.) So one role the summer school served was just to introduce a bunch of Chinese AI students to some allied disciplines -- neuroscience, cognitive psychology, philosophy of mind -- that they hadn't seen much during their formal education so far.

(Actually, separately from the Summer School, I did give a talk to some undergrads in the Software Engineering School, on AI and Gaming, which contained one funny bit (unfortunately that talk was not videotaped). I wasn't sure if the students understood what I was talking about, so as a test I showed them this picture as part of my powerpoint


Normally I use this lovely picture as an example of "conceptual blending" (a cognitive operation that OpenCog and other AGI systems must carry out), but this time I announced it differently; I said: "Furthermore, the Artificial Brain Lab here at XMU has an ambitious backup plan, in case our computer science approach to AGI fails. We've devised a machine that can remove the head from a graduate student, and attach it to the body of a Nao humanoid robot, and thus create a kind of synergetic cyborg intelligence." I was curious to see if these Chinese undergrads understood my English well enough to tell that I was joking -- but from their reaction, I was unable to tell. They laughed because the picture looked funny, but I still don't know if they understood what I was saying! Fortunately the Summer School students were less inscrutable, and more reactive and communicative! And overall the AI in Games lecture went well in spite of this perplexing crosscultural joke experience....)

(As an aside within an aside, I also learned during various conversations that typical Chinese high school students spend from 7AM till 10PM or so at school, 6 days a week. Damn.)

Another thing that surprised me was the strength of knowledge the Chinese students had in neural nets, fuzzy logic, computer vision and other "soft computing" and robotics related AI, as compared to logic-based AI. By and large, they had a very strong mathematics background, and a good knowledge of formal logic -- but fairly little exposure to the paradigm in which logic is applied to create AI systems. Quite different from the typical American AI education.

All in all the Chinese seemed to have a lot less skepticism about "strong AI" than Americans. It's not that they had a great faith in its immediacy -- more that they lacked the egomaniacal confidence in its extreme difficulty or implausibility, which one so often finds in Westerners. Chinese culturally seem much more comfortable with accepting situations of great unconfidence, in which the evidence just doesn't exist to make a confident estimate.

I came to the summer school from the Toward a Science of Consciousness conference in Hong Kong, where I led a Machine Consciousness workshop -- which I won't write about here, because I wrote a summary of it for H+ magazine, which will appear shortly. Issues of machine consciousness came up now and then at the summer school, but interestingly, they seem to hold a lot less fascination for Chinese than for Westerners. When I put forth my panpsychist perspective in China (that the universe as a whole is conscious in a useful sense, and different systems -- like human brains and digital computers -- manifest this consciousness in different ways ... and our "theater of reflective consciousness" is one of the ways universal consciousness can manifest itself in certain sorts of complex systems), no one really bats an eye (and not just because the Chinese lack a taste for eye-batting). Not that Chinese scientists consider this panpsychist perspective wholly obvious or necessarily correct; but nor do they consider it outrageous -- and, most critically, very FEW Chinese seem to feel like many Westerners do, that "reductionism" or "materialism" is obviously correct. Once you remove the tendency toward dogmatic materialism, the whole topic and dilemma of "machine consciousness" loses its bite....

China versus California (A Semi-Digression)

(This section contains some ramblings on Oriental versus California culture, and the Singularity -- which are only semi-relevant to the summer school, but I'll put them here anyway, because I find them amusing! Hey, this is a blog, anything goes ;-)

In mid-July I voyaged from the Xiamen AGI summer school to California where I gave the keynote speech at the IJCAI workshop on Neural-Symbolic computing (a really interesting gathering, which I'll discuss some other time), and then gave a lecture on AGI at the Singularity University (at NASA Ames Lab, in Silicon Valley).

The contrast between the SU students and the Chinese AGI Summer School students couldn't have been more acute.

For one thing, there was a huge contrast of ego ... to phrase things dramatically: The SU students emanated an attitude that seemed to say "We know more than anyone on the planet!! We already knew almost everything we need to know to dominate the world as part of the techno-elite!"

The Chinese students were not actually more ignorant (though their knowledge bases had different strengths and weaknesses than those of the SU students), but they were dramatically more humble about their state of knowledge!

The SU students also seemed extremely eager to project everything I said about AGI into the world they knew best: Silicon Valley style Internet software. So, most of the questions during and after my talk centered around the theme: "Isn't it unnecessary to work on AGI explicitly ... won't AGI just emerge from the Internet after Silicon Valley startup firms create enough cool narrow-AI online widgets?" When I said I thought this was unlikely, then the questions turned to: "OK, but rather than writing an AGI that actually thinks on its own, shouldn't you just write a narrow-AI that figures out the best way to combine existing online widgets, and achieves general intelligence that way?" And so forth.

But I don't want to make it sound like the SU student body is "all of one mind" -- it's certainly a heterogeneous bunch. At the lunch following my talk at SU, one SU student surprised me with the following statement (paraphrased): "One reason I think AI systems may not achieve the same kind of ethical understandings or states of mind as humans, is that they lack one of the most important human characteristics: our humbleness. We humans have a lot of limitations in our bodies and minds, and these limitations have made us humble, and this humbleness is part of what makes us ethical and part of what makes us profoundly intelligent in a way that a mere calculating machine could ever be."

I laughed out loud and immediately said to the student: "OK, I'm onto you. You're not American." (The student did look Asian ... but I was guessing he was not Asian-American.)

He admitted to being from Korea ... and I noted that few Americans -- and especially no Silicon Valley techno-geek -- would ever identify humbleness as a central characteristic of humans or a key to human intelligence!

Then I couldn't help thinking of the saying "Pride comes before a fall" ... and Vinge's (correct) characterization of the Singularity as a point after which HUMANS WILL HAVE NO IDEA WHAT'S GOING ON ... i.e. no real ability to predict what happens next, as superhuman nonhuman intelligences will be dominating the scene.

Philosopher Walter Kauffmann coined the dorky but evocative term "humbition" to denote the combination of humility and ambition. There's not much humbition in Silicon Valley ... nor for that matter in the public trumpetings of the Chinese government ... but there was a LOT of humbition in the Chinese students at the AGI summer school and the Artificial Brain Lab. Perhaps this quality will serve them well as the world advances, and our knowledge and intuitions prove decreasingly adequate to comprehend our situation...

If you believe that AGI will be created from piecing together narrow-AI internet widgets, then yeah, mostly likely AGI will be created by the Silicon Valley techno-elite. But if (as I suspect) it requires fundamentally different ideas from the ones now underlying the world's technological infrastructure ... maybe it will be created by people who are more open to fundamentally new and different ideas.

But this leads into the next blog post I'm going to write, exploring the question of whether Hugo de Garis is right that AGI is going to get created in China rather than the West!

Musings on the Concept of a Systematic AGI Curriculum, and Lessons for Future AGI Summer Schools

Next, what did I learn this summer about the notion of an AGI summer school, and about teaching AGI altogether?

One big lesson that got reinforced in my mind is: Teaching AGI is very different than teaching Narrow AI!

There is basically no systematic AGI education in universities anywhere on the planet, and this fact certainly helps to perpetuate the current AGI research situation (in which there is very little AGI research going on). By and large, everywhere in the world, students graduate with PhD degrees in AI, without really knowing what "AGI research" means.

Another conclusion I came to is that a carefully crafted "AGI Summer School" curriculum could play a major role -- not only in providing AGI education, but in demonstrating how AGI material should be structured and taught.

However, creating a thorough, systematic AGI curriculum would be a lot of work ... and we didn't really attempt it for the First AGI Summer School. I think the lectures mostly went very well this time (well, you can judge when the videos come online!!), and the sequencing of the lectures made good didactic sense -- but, for the next AGI summer school, we'll put a little more thought into framing the curriculum in a systematic way. Now, having done the summer school once, it's more clear to me (and probably the other participants as well) what an AGI curriculum should be like.

First of all, it's obvious that to make a systematic AGI curriculum, one would need some systematic background curriculum in areas like

• Neuroscience
• Linguistics
• Philosophy of Mind
• Psychology (of Cognition, Perception, Emotion, etc.)

We didn't do enough of that this time. Allan Combs' lectures at the Xiamen summer school formed a nice start toward a Neuroscience background curriculum for AGI, but due to lack of time he couldn't do everything needed.

In this vein, one thing that became clear to me at the Xiamen summer school is: The standard "cognitive science" curriculum would certainly fill this need for background, but it's not exactly right, because it's not specifically focused on AGI ... AGI students really only need to digest a certain subset of the cognitive science curriculum, selected specifically with AGI-relevance in mind. But judiciously making this selection would be a nontrivial task in itself.

Next, as part of a thorough AGI curriculum, one would need a systematic review of different conceptions of what "general intelligence" is -- we did such a review at the Xiamen summer school, but not all that systematically. Pei Wang gave a nice talk on this theme, and then Joscha Bach and I presented our own conceptions of GI, and I also briefly reviewed the Hutter/Schmidhuber "universal intelligence" perspective.

Then there's the matter of reviewing the various AGI architectures out there. I think the Xiamen summer school did a fairly good job of that, with in-depth treatments of OpenCog, Pei Wang's NARS architecture, and Joscha Bach's MicroPsi ... and a briefer discussion of Hugo de Garis's neural net based Artificial Brain approach ... and then very quick reviews of other AGI architectures like SOAR and LIDA. Of course there are many, many architectures one could discuss, but in a limited time-frame one has to pick just a few and focus on them. (It would be nice if there were some more systematic way to review the various AGI architectures out there than taking a "laundry list" approach, but this isn't an education problem, it's a fundamental theory problem -- no such systematization exists, even in the research literature.)

There were a lot of OpenCog-related lectures at the Xiamen summer school, and one thing I felt was that it was both too much and too little! Too much OpenCog for a generic AGI Summer School, but too little for a real in-depth OpenCog education. At future summer schools we may split OpenCog stuff off to a greater extent: give a briefer OpenCog treatment in the main summer school lectures, and then do a separate one-week OpenCog lecture series after that, for students who want to dig deep into OpenCog.

Another educational issue is that each AGI architecture involves different narrow-AI algorithms, so that to really follow the architecture lectures fully, students needed to know all about forward and backward inference, attractor, feedforward and recurrent neural nets, genetic algorithms and genetic programming, and so forth. (Most of them did have this knowledge, so it wasn't a problem; actually this might be more of a problem in the US than in China, as China's education system is very strong on comprehensively teaching factual knowledge.) That is: even though AGI is quite distinct from narrow AI, existing AGI architectures make ample use of narrow-AI tools, so students need a good grounding in narrow AI to grok current AGI systems. It would be good to make a systematic list of tutorials on the most AGI-relevant areas of narrow AI, for students whose narrow-AI background is spotty. Again, we did some of this for the Xiamen summer school, but probably not enough.

Finally there's the terminology issue. There is no good "AGI glossary", and every researcher uses terms in slightly different ways. Updating and enlarging an online AGI glossary would be a great project for students at an AGI summer school to participate in!

Undramatic Non-Conclusion

So, the first AGI Summer School went pretty interestingly, and I'm really glad it happened. It was interesting to get to know China a little bit, and to get some experience teaching AGI in an intensive-course context. I learned a lot, and I guess the other faculty and the students did too.... I also made a number of excellent new friends, both among the Chinese and the foreign students. As with many complex real-world experiences, I don't really have any single dramatic summary or conclusion to draw ... but I'm looking forward both to future AGI summer schools, and to future experiences with "AGI in China"....

Quantum Logic Models of Classical Systems Give a New Twist on Quantum Consciousness

A while ago I wrote a blog post suggesting that quantum logic should be applied more generally than to quantum physical systems ... that it should be applied to complex classical systems in some cases as well, if they are so complex that their states are unobservable to a certain observer.  

This, I suggested, would require making the choice of logic observer-dependent: i.e., the system T might best be modeled by system S using quantum logic, but by system R using classical logic.

I didn't at the time see how to make this speculation rigorous but I've now found a related literature that helps a lot.

And by refining my previous idea, I've come up with an argument that possibly human consciousness may be effectively modeled using quantum logic, whether or not the human brain is a quantum system.

I may write a paper on this stuff at some point (in which process I'll probably figure out nicer ways to express the ideas), but wanted to write it down now while it's fresh in my mind.

Atmanspacher's Idea

Diederik Aerts and Liane Gabora have written some very nice papers related to this topic ... and I read their stuff years ago but didn't quite see how to connect it to my relevant intuitions.   

What I discovered just recently was the related work of Harald Atmanspacher, which ties in more directly with the way I was thinking about these issues.  (Some relevant papers by both of these guys are linked to at the end of this post.)

Put simply, Atmanspacher's view is that: In any case where two properties of a system cannot be simultaneously measured with high accuracy, you have a situation that should be modeled using quantum logic.

I.e., quantum logic should be applied to any case where there are incompatible observables ... whether or not this is due to quantum microphysics.

Making the Choice of Logic Observer-Dependent

My twist on Atmaspacher's idea is to suggest that quantum logic should be applied, by a cognitive system, to any situation that has two aspects which (perhaps by quantum microphysics, or perhaps simply due to its limitations as a cognitive system) the cognitive system cannot model simultaneously. 

That is: If T has two aspects, and S cannot model these two aspects of T simultaneously without becoming non-S, then from the perspective of S, these aspects of T should be modeled using quantum logic rather than classical logic.

Note that S in this argument is not a specific physical system at a particular point in time, but rather a category of instantaneous physical systems, which are being considered as instantiations of a single abstract "system" (for example, "Ben Goertzel" is a category of instantaneous physical systems).  

So, my suggestion is that whether T should be reasoned about by quantum or classical logic, must be determined by relativizing the reasoning to some category of instantaneous physical systems.

Possible Implications for Quantum Consciousness

What spurred me to start digging into these issues just now was a conversation with Stuart Hameroff, who believes consciousness to be a quantum phenomenon.  

My suggestion is: It could be that a quantum model of human consciousness is the right one, even if the underlying physics of the brain is basically "classical" (and I don't claim to know for sure whether it is or not).

(Note that I referred above to a classical model of "human consciousness", not of consciousness in general -- I tend toward panpsychism, meaning I think everything is conscious and different systems just manifest universal consciousness in different ways.)

As a natural consequence of the above argument, I would suggest that each of us individually, due to our own processing limitations, cannot view ourselves in all aspects simultaneously. 

If this is true, then perhaps we should model ourselves using quantum logic.  

Being panpsychist I would not identify this with consciousness, but I would say that systems which are sufficiently complex that they implicitly model themselves using quantum logic, in predicting and analyzing their own dynamics, presumably have a distinctive character to the way they manifest universal consciousness.

Can We Tell the Cause of the Incompatibility?

An interesting question is: if I, as a cognitive system, am confronted with incompatible observables ... in what sense can I tell what the cause of this incompatibility is?

Can I tell a case where the incompatibility is caused by my own cognitive limitations, from a case where it is caused by fundamental indeterminacy such as is sometimes hypothesized to occur in quantum microphysics?

It would seem there is no direct way to make this determination, but we can induce general theories from observations of other system aspects, which lead us to hypotheses regarding the causes of an incompatibility.

Some Nice Quotes on Quantum Modeling of Classical Systems

These quotes come from

http://www.quantum-cognition.de/

Andrei Khrennikov:

“I propose to consider any system which produces quantum statistics as quantum (”quantum-like”). A possible test is based on the interference of probabilities. I was mainly interested in using such an approach to ”quantumness” to extend the domain of applications of quantum mathematical formalism and especially to apply it to cognitive sciences. There were done experiments on interference of probabilities for ensembles of students and a nontrivial interference was really found. … Yes, we might expect nonclassical statistics, but there was no reason to get the quantum one, i.e., cos-interference. But we got it!”

Diederik Aeerts & Liane Gabora:

"While some of the properties of quantum mechanics are essentially linked to the nature of the microworld, others are connected to fundamental structures of the world at large and could therefore in principle also appear in other domains than the micro-world."

Diederik Aeerts:

"The emergence of quantal macrostates does not necessarily require the reference to corresponding quantal microstates" 

Harald Atmanspacher, Hans Primas & Peter beim Graben:

"A generalized version of the formal scheme of ordinary quantum theory, in which particular features of ordinary quantum theory are not contained, should be used in some non-physical contexts."

"Complementary observables can arise in classical dynamic systems with incompatible partitions of the phase space."

A Few Relevant References

Atmanspacher:

"Weak quantum theory"

http://www.igpp.de/english/tda/pdf/wqt.pdf

"Complementarity in Bistable Perception"

http://unjobs.org/authors/harald-atmanspacher

Aerts:

At

http://www.vub.ac.be/CLEA/aerts/publications/

see

• Aerts, D. (1982). Example of a macroscopical situation that violates Bell inequalities. Lettere al Nuovo Cimento, 34, pp. 107-111. 
  
• Aerts, D. (1991). A mechanistic classical laboratory situation violating the Bell inequalities with 2sqrt(2), exactly 'in the same way' as its violations by the EPR experiments. Helvetica Physica Acta, 64, pp. 1-23. 
  
• Aerts, D. and Gabora, L. (2005). A theory of concepts and their combinations I: The structure of the sets of contexts and properties. Kybernetes, 34, pp. 167-191.
  
• Aerts, D. and Gabora, L. (2005). A theory of concepts and their combinations II: A Hilbert space representation. Kybernetes, 34, pp. 192-221
  

How many transhumanists does it take to change a light bulb?

Infinity.... None of them will touch the light bulb at all; they'll all just sit around talking amongst themselves and waiting for someone else to invent a self-changing cyber light bulb.

Reinforcement Learning: Some Limitations of the Paradigm

(This email summarizes some points I made in conversation recently with an expert in reinforcement learning and AGI. These aren't necessarily original points -- I've heard similar things said before -- but I felt like writing them down somewhere in my own vernacular, and this seemed like the right place....)

Reinforcement learning, a popular paradigm for AI, economics and psychology, models intelligent agents as systems that choose their actions in such a way as to maximize their future reward. There are various ways of averaging future reward over various future time-points, but all of these implement the same basic concept.

I think this is a reasonable model of human behavior in some circumstances, but horrible in others.

And, in an AI context, it seems to combine particularly poorly with the capability for radical self-modification.


Reinforcement Learning and the Ultimate Orgasm

Consider for instance the case of a person who is faced with two alternatives

• A: continue their human life as would normally be expected
• B: push a button that will immediately kill everyone on Earth except them, but give them an eternity of ultimate trans-orgasmic bliss

Obviously, the reward will be larger for option B, according to any sensible scheme for weighting various future rewards.

For most people, there will likely be some negative reward in option B ... namely, the guilt that will be felt during the period between the decision to push the button and the pushing of the button. But, this guilt surely will not be SO negative as to outweigh the amazing positive reward of the eternal ultimate trans-orgasmic bliss to come after the button is pushed!

But the thing is, not all humans would push the button. Many would, but not all. For various reasons, such as love of their family, attachment to their own pain, whatever....

The moral of this story is: humans are not fully reward-driven. Nor are they "reward-driven plus random noise".... They have some other method of determining their behaviors, in addition to reinforcement-learning-style reward-seeking.


Reward-Seeking and Self-Modification: A Scary Combination

Now let's think about the case of a reward-driven AI system that also has the capability to modify its source code unrestrictedly -- for instance, to modify what will cause it to get the internal sensation of being rewarded.

For instance, if the system has a "reward button", we may assume that it has the capability to stimulate the internal circuitry corresponding to the pushing of the reward button.

Obviously, if this AI system has the goal of maximizing its future reward, it's likely to be driven to spend its life stimulating itself rather than bothering with anything else. Even if it started out with some other goal, it will quickly figure out to get rid of this goal, which does not lead to as much reward as direct self-stimulation.

All this doesn't imply that such an AI would necessarily be dangerous to us. However, it seems pretty likely that it would be. It would want to ensure itself a reliable power supply and defensibility against attacks. Toward that end, it might well decide its best course is to get rid of anyone who could possibly get in the way of its highly rewarding process of self-stimulation.

Not only would such an AI likely be dangerous to us, it would also lead to a pretty boring universe (via my current aesthetic standards, at any rate). Perhaps it would extinguish all other life in its solar system, surround itself with a really nice shield, and then proceed to self-stimulate ongoingly, figuring that exploring the rest of the universe would be expected to bring more risk than reward.

The moral of the above, to me, is that reward-seeking is an incomplete model of human motivation, and a bad principle for control self-modifying AI systems.


Goal-Seeking versus Reward-Seeking

Fortunately, goal-seeking is more general than reward-seeking.

Reward-seeking, of the sort that typical reinforcement-learning systems carry out, is about: Planning a course of action that is expected to lead to a future that, in the future, you will consider to be good.

Goal-seeking doesn't have to be about that. It can be about that ... but it can also be about other things, such as: Planning a course of action that is expected to lead to a future that is good according to your present standards.

Goal-seeking is different from reward-seeking because it will potentially (depending on the goal) cause a system to sometimes choose A over B even if it knows A will bring less reward than B ... because in foresight, A matches the system's current values.


Non-Reward-Based Goals for Self-Modifying AI Systems

As a rough indication of what kinds of goals one could give a self-modifying AI, that differ radically from reward-seeking, consider the case of an AI system with a goal G that is the conjunction of two factors:

• Try to maximize the function F
• If at any point T, you assess that your interpretation of the goal G at time T would be interpreted by your self-from-time-(T-S) as a terrible thing, then roll back to your state at time S

I'm not advocating this as a perfect goal for a self-modifying AI. But the point I want to make is this kind of goal is something quite different from the seeking of reward. There seems no way to formulate this goal as one of reward maximization. This is a goal that involves choosing a near-future course of action to maximize a certain function over future history -- but this function is not any kind of summation or combination of future rewards.


Limitations of the Goal-Seeking Paradigm

Coming at the issue from certain theoretical perspectives, it is easy to overestimate the degree to which human beings are goal-directed. It's not only AI theorists and engineers who have made this mistake; many psychologists have made it as well, rooting all human activity in goals like sexuality, survival, and so forth. To my mind, there is no doubt that goal-directed behavior plays a large role in human activity -- yet it also seems clear that a lot of human activity is better conceived as "self-organization based on environmental coupling" rather than as explicitly goal-directed.

It is certainly possible to engineer AI systems that are more strictly goal-driven than humans, though it's not obvious how far one can go in this direction without sacrificing a lot of intelligence -- it may be that a certain amount of non-explicitly-goal-directed self-organization is actually useful for intelligence, even if intelligence itself is conceived in terms of "the ability to achieve complex goals in complex environments" as I've advocated.

I've argued before for a distinction between the "explicit goals" and "implicit goals" of intelligent systems -- the explicit goals being what the system models itself as pursuing, and the implicit goals being what an objective, intelligent observer would conclude the system is pursuing. I've defined a "well aligned" mind as one whose explicit and implicit goals are roughly the same.

According to this definition, some humans, clearly, are better aligned than others!

Summary & Conclusion

Reward-seeking is best viewed as a special case of goal-seeking. Maximizing future reward is clearly one goal that intelligent biological systems work toward, and it's also one that has proved useful in AI and engineering so far. Thus, work within the reinforcement learning paradigm may well be relevant to designing the intelligent systems of the future.

But, to the extent that humans are goal-driven, reward-seeking doesn't summarize our goals. And, as we create artificial intelligences, there seems more hope of creating benevolent advanced AGI systems with goals going beyond (though perhaps including) reward-seeking, than with goals restricted to reward-seeking.

Crafting goals with reasonable odds of leading self-modifying AI systems toward lasting benevolence is a very hard problem ... but it's clear that systems with goals restricted to future-reward-maximization are NOT the place to look.

Science-synergetic philosophy: the religion of the future?

(This may seem a hackneyed topic, but there are some moderately original points near the end here, if you bear with me ...)

As a card-carrying, future-thinking transhumanist, I take it as obvious that most of the particulars of current religions are relics of earlier eras in human cultural development, which currently do a lot of harm along with doing some good.

But I still find it interesting to ask what aspects of religion reflect underlying phenomena that are essential, meaningful and necessary -- and are likely to continue as humanity transcends the traditional "human condition" and enters its next phase of development....


Fish and Eagleton on the Wonders of Theology

What spurred this blog post was: My dad pointed out to me this New York Times blog post by Stanley Fish reviewing a book that extols the merits of religion (Reason, Faith and Revolution by Terry Eagleton).

The basic point Fish makes is that religion offers something science by its very nature cannot.



Eagleton acknowledges ... many terrible things have been done in religion’s name — but at least religion is trying for something more than local satisfactions, for its “subject is nothing less than the nature and destiny of humanity itself, in relation to what it takes to be its transcendent source of life.”


He notes that science cannot address what he calls "theological questions", where


By theological questions, Eagleton means questions like, “Why is there anything in the first place?”, “Why what we do have is actually intelligible to us?” and “Where do our notions of explanation, regularity and intelligibility come from?”


He also notes that the author is


... angry, I think, at having to expend so much mental and emotional energy refuting the shallow arguments of school-yard atheists like Hitchens and Dawkins.


I haven't read Eagleton's book and I'm unlikely to do so -- I have a long list of more interesting-looking reading material -- but Fish's summary did resonate with a paper I'm in the middle of writing (it's paused while I work on more urgent stuff) on the limits of science.

My basic point in that paper will be a simple one: science is based on finite sets of finite-precision observations. That is, all of scientific knowledge is based on some finite set of bits, comprising the empirical observations accepted by the scientific community.

To extrapolate beyond this bit-set, some kind of assumption is needed. To put it another way, some kind of "faith" is needed. Hume was the first one to make this point really clearly ... and we now understand the "Humean problem of induction" well enough to know it's not the kind of thing that can be "solved."

The Occam's Razor principle tries to solve it -- it says that you extrapolate from the bit-set of known data by making the simplest possible hypothesis. This leads to some nice mathematics involving algorithmic information theory and so forth. But of course, one still has to have "faith" in some measure of simplicity!

So: doing or using science requires, in essence, continual acts of faith (though these may be unconscious and routinized rather than conscious and explicit). To the extent that Dawkins, Hitchens or other anti-religion commentators de-emphasize this point, they're engaging in judicious marketing. (It's hard for me to feel too negative toward them about this, however, given the far more explicitly and dramatically dishonest marketing that religion has carried out over the last millennia.)

My paper will focus on what the limits of science tell you about AI, machine consciousness and so forth -- and I'll save that for another blog post, or the paper itself. (Don't worry though, my conclusion is not that scientifically enginering AGI is impossible ... I haven't lost the faith!)

Anyway, I certainly agree with Fish and Eagleton that religion addresses very important questions that science cannot, by its nature, answer.

But I find it rather screwy that Eagleton refers to


“Why is there anything in the first place?”, “Why what we do have is actually intelligible to us?” and “Where do our notions of explanation, regularity and intelligibility come from?”


and so forth as theological questions.

Surely, these are philosophical questions.

One can answer them in various ways without invoking any deities or demons!

"Why does God exist?" is a theological question ...

"Why does anything exist?" is philosophical...

(Though, for the record, I don't think "Why does anything exist?" is a very useful philosophical question. I'm more interested in questions like

• "Why do separate objects exist, instead of just one big fluid cosmic mass?"
• "In what sense could the universe be considered compassionate?"
• "How much ethical responsibility should I feel toward (which) other minds?"
  
• "Why does my mind perceive such a small subset of the space of all possible patterns?"
• "How much can a mind grow and expand without losing its sense of self and becoming, experientially, a 'fundamentally different being'?"
• "What is it like to be a rock?"
• etc.

)

Theology is one way of providing answers to philosophical questions ... but by no means the only way.

I think that religion addresses some very important questions, that are beyond the scope of science -- and by and large provides these questions with extremely bad answers.

One of the many limitations of religion as conventionally conceived is indicated by the quote, given above, that religion's


“subject is nothing less than the nature and destiny of humanity itself....”


From a transhumanist perspective, the qualifier "nothing less than" is misplaced, as this is actually a very limiting subject. The nature and destiny of humanity are important; but one of the things that science has opened our minds to is the relative insignificance of humanity in the space of possible minds. I'm more interested in philosophies that address the nature and destiny of mind itself, rather than just the nature and destiny of one species on one planet.

It is of course a subtle matter to compare and judge different explanations to philosophical questions. You can't compare them using scientific or mathematical methods ... and of course the question of how to evaluate philosophical views becomes "yet another tough philosophical question", tied in with all the other ones.

A crude way to say it, is that it comes down to an intuitive judgment ... which leads into questions of how one can refine and improve one's intuition ... and these questions, of course, possess numerous answers that are philosophical- or religious- tradition -dependent...


Science-synergetic philosophy


It does seem to me, though, that there is an interesting notion of science-synergetic philosophy lurking somewhere in all this.

Suppose we take for granted that doing science -- just like other aspects of living life -- relies on a constant stream of acts of faith, which can't be justified according to science....

One may then note that there are various systems for mentally organizing these acts of faith.

Religions are among them. But religions are quite detached from the process of doing science.

It seems sensible to think about philosophical systems -- i.e. systems for organizing inner acts of faith -- that are intrinsically synergetic with the scientific process. That is, systems for organizing acts of faith, that

• when you follow them, help you to do science better
• are made richer and deeper by the practice of science
  

One can broaden this a little and think about philosophical systems that are intrinsically synergetic with engineering and mathematics as well as science.

Now, one cannot prove scientifically that a "scientifically synergetic philosophy" is better than any other philosophy. Philosophies can't be validated or refuted scientifically.

So, the reason to choose a scientifically synergetic philosophy has to be some kind of inner intuition; some kind of taste for elegance, harmony and simplicity; or whatever.

One prediction I have for the next century is that scientifically synergetic philosophies will emerge into the popular consciousness and become richer and deeper and better articulated than they are now.

Because Fish and Eagleton are right about some things: people do need more than science ... they do need collective processes focused on the important philosophical questions that go beyond the scope of science.

But my prediction is that we are going to trend more toward philosophical systems that are synergetic with science, rather than ones that co-exist awkwardly with science.

What will these future philosophical systems be like?

There's nothing extremely new about the concept of science-synergetic philosophy, of course.

Plenty of non-religious scientists and science-friendly non-scientists have created personal philosophies that don't involve deities or other theological notions, yet do involve meaningful approaches to personally exploring the "big questions" that religions address.

Among the many philosophers to take on the task of creating comprehensive science-synergetic philosophical systems, perhaps my favorite is Charles Peirce (who also developed a nice philosophy of science, though one that IMO is significantly incomplete ... but I've discussed that elsewhere.)

Building on work by Peirce and loads of others, I tried to lay out a science-synergetic philosophical system in my book The Hidden Pattern -- but like Peirce's writings, that is a fairly academic work, not an informal tract designed to inspire the common human in their everyday life.

My friend Philippe van Nedervelde likes to talk about this sort of thing as a "TransReligion/ UNReligion", but I confess to not finding that terminology very compelling.

Philippe is interested in (among many other things!) developing vaguely religion-like rituals that coincide with some sort of science-synergetic philosophy. There has been talk about formulating a "TransReligion/ UNReligion" as an outgrowth of the futurist group now called "The Order of Cosmic Engineers." Which I think is an interesting idea ... yet I'm not really sure it's the direction things will (or should) go.

I'm not sure there will emerge any one "Bible of science-synergetic transhumanist philosophy" ... nor any science-synergetic-philosophy analogues of speaking in tongues, kneeling at the altar, or consuming the simulated blood and flesh of the Savior the Son of God who gave his life for our sins. Perhaps, science-synergetic philosophy may wind up being something that pervades human culture in more of a broad-based, implicit way.

Time will tell!

Does Long-Lived Quantum Coherence Underlie Biological Processes?

Is the human brain, at the levels directly relevant for analysis of cognition, best modeled as a classical or quantum system?

(For instance, a baseball in some sense needs to be modeled as a quantum system -- in the sense that the way its molecules hold together can be described only using quantum not classical physics; but classical physics can be used to explain the normally relevant aspects of its macroscopic behavior. So at the levels directly relevant for analysis of a baseball game, a baseball is best modeled as a classical system. OTOH, at the levels directly relevant for analysis of the electromagnetic behavior of a Superconducting Quantum Interference Device (SQUID) -- a small but macroscopic device, used in magnetoencephalography machines, and demonstrating macroscopic quantum coherence in its magnetic field -- the SQUID is best modeled as a quantum system. Classical physics models just won't explain why the SQUID, a device you can hold and pinch between your fingers (though it only works when supercooled, which would freeze your fingers!), makes MEG machines work.)

Current brain theory indicates that for understanding its role in giving rise to the mind, the brain is most effectively modeled as a classical system (i.e. the brain is more like a baseball than a SQUID) ... but of course current brain theory could be incomplete.

(Even if the brain is a macroscopic quantum system, this of course doesn't prove that quantum dynamics are necessary for intelligence or consciousness or anything like that. Those are bigger and deeper questions, and I've argued in the past that sufficiently complex "classical" systems might need to be treated using quantum logic ... but this gets into a lot of deep issues that I don't want to digress onto here.)

Stuart Hameroff is one of the more vocal proponents of the "quantum brain" idea, and he has a new paper reporting a new theory in this direction, arguing that dendro-dendritic synapses are mediated via macroscopic quantum dynamics, thus posing a quantum neural net that operates in complex coordination with the classical neural net formed by axonal-dendritic synapses.

I don't have a strong opinion on that particular theory of Hameroff's. I look forward to discussing it with him at the Toward a Science of Consciousness conference in Hong Kong next month.

But I was struck by one of the references at the end of his paper, a Nature paper entitled

Evidence for wavelike energy transfer through quantum coherence in photosynthetic systems

This is a 2007 paper that I had not noticed before, and it's interesting because it gives solid evidence of macroscopic quantum coherence in a biological process.

To quote part of the abstract:


Here we extend previous two-dimensional electronic spectroscopy investigations of the FMO bacteriochlorophyll complex, and obtain direct evidence for remarkably long-lived electronic quantum coherence playing an important part in energy transfer processes within this system. The quantum coherence manifests itself in characteristic, directly observable quantum beating signals among the excitons within the Chlorobium tepidum FMO complex at 77 K. This wavelike characteristic of the energy transfer within the photosynthetic complex can explain its extreme efficiency, in that it allows the complexes to sample vast areas of phase space to find the most efficient path.

In the comments to an earlier edit of this blog post, someone pointed out this more recent paper

Quantum Zeno Effect Underpinning the Radical-Ion-Pair Mechanism of Avian Magnetoreception

whose abstract says

The intricate biochemical processes underlying avian magnetoreception, the sensory ability of migratory birds to navigate using earths magnetic field, have been narrowed down to spin-dependent recombination of radical-ion pairs to be found in avian species retinal proteins. The avian magnetic field detection is governed by the interplay between magnetic interactions of the radicals unpaired electrons and the radicals recombination dynamics. Critical to this mechanism is the long lifetime of the radical-pair spin coherence, so that the weak geomagnetic field will have a chance to signal its presence. It is here shown that a fundamental quantum phenomenon, the quantum Zeno effect, is at the basis of the radical-ion-pair magnetoreception mechanism. The quantum Zeno effect naturally leads to long spin coherence lifetimes, without any constraints on the systems physical parameters, ensuring the robustness of this sensory mechanism. Basic experimental observations regarding avian magnetic sensitivity are seamlessly derived. These include the magnetic sensitivity functional window and the heading error of oriented bird ensembles, which so far evaded theoretical justification. The findings presented here could be highly relevant to similar mechanisms at work in photosynthetic reactions. They also trigger fundamental questions about the evolutionary mechanisms that enabled avian species to make optimal use of quantum measurement laws.

This of course is even more intriguing than the green sulphur bacteria stuff, because it has to do with perception in an intelligent macroscopic animal.

Hameroff's point in citing the paper on green sulphur bacteria (and it's a good one) seems to be: if long-lived quantum coherence can play an important role in photosynthesis, couldn't it also play a role in the brain somehow ... e.g. maybe via dendro-dendritic synaptic gap junctions?

The extrapolation from these other results to neuroscience is speculative, sure.... But this kind of result does make the possibility of quantum coherence impacting human cognition seem a bit less fanciful.

After all, I often recall that in the late 90's all the neuroscientists I talked to told me there was no neurogenesis nor synaptogenesis in adult mammals. Oops. Now they've got new data and changed their mind. My point isn't that quantum coherence is related to neuro or synapto genesis (though, who knows...), but rather that neuroscientists -- simultaneously with displaying the usual humility of biologists regarding the complexity of the systems they're studying -- have a long-standing habit of assuming the concept-set underlying their current understanding is much more adequate than it really is.

Our ignorance of the brain is why my own AI work is not based on trying to closely model the brain. Of course, it's possible that intelligence is fundamentally based on some freaky neuroquantum phenomenon, so that all digital-computer AI work is doomed by some intrinsic limitations ... but I doubt it. My own guess is that, even if the brain does involve macroscopic quantum coherence in some interesting sense, one can still make transhumanly intelligent systems using digital computers. And of course, if this doesn't work -- or if these transhumanly intelligent systems turn out to lack some crucial aspect of self-awareness as the quantum-consciousness advocates argue -- then we can always add some funky quantum computing chips into our AGI server farm!

"Transcendent Man" ... my brush with movie-stardom ... and my answer to "What is Your Goal?"

My review of the Kurzweil-biopic/futurist-think-piece documentary Transcendent Man -- which features me mouthing off for 4-5 minutes in a zebra-striped cowboy hat -- in HPlus Magazine is here:

http://www.hplusmagazine.com/articles/ai/transcendent-man-film-about-kurzweil

In that article, as well as reviewing the film, I also recount some moderately interesting dialogue btw me and Ray Kurzweil that occurred in the moderated discussion at the end of the film's premiere at the Tribeca Film Festival...

After that conversation with Ray I discuss at the end of the article, the discussion-moderator asked me another question (which I didn't put in the review article): he asked me what my goal was. What was I trying to achieve?

What I said was something like this: "I would like me, and any other human or nonhuman animal who wants to, to be able to increase our intelligence and wisdom gradually ... maybe, say, 37.2% per year ... so that we can transcend to higher planes of being gradually and continuously, and feel ourselves becoming gods ... appreciate the process as it unfolds."

That's what I'm after, folks. Hope you'll come along for the ride!

Teaching Dolphins Lojban ... Giving Dolphins Prosthetic Hands

A follow-up to my prior posts on cetacean intelligence...

I thought a bit about innovative ways we might be able to communicate better with our cetacean planet-mates...

1. Teach Dolphins Lojban

A couple decades ago, efforts were made to teach dolphins simple English, without dramatic success. Discussions were also had regarding creation of some sort of species-independent interlingua, which humans and dolphins could use to communicate with each other.

It occurred to me that using Lojban for that interlingua could make sense. Potentially, one could create special Lojbanic vocabulary for the shared human/dolphin environment. Lojban grammar is simple and unambiguous, and certainly has less species-specificity than any human natural language.

Also, one could create a form of Lojban "phonology" that generally follows the sound-production patterns habitually by dolphins, and speak to dolphins in this "Delphic Lojban" alongside the usual "human Lojban."

The biggest disadvantage of this approach is that it requires some human cetaceologists to learn Lojban.... But this cost seems worth paying, as the odds of success seem much higher than with human natural languages.

Note that there is no straightforward way to make a "phonologically Delphic" version of English. But because Lojban syntax is just a linearization of logical relationships, one could make a Delphic version of Lojban by translating those same logical relationships into sound in a wholly different way than is done in the human version of Lojban.


2. Give Dolphins Prosthetic Hands

Inside a dolphin's flippers, are bones that look like they should correspond to claws or fingers.

What if we created prosthetic fingers and thumbs for dolphins, and connected them to these bones ... and also connected them to the dolphin nervous system?

Admittedly, these modified dolphins would suffer impaired swimming ability, though one would hope the degree of this phenomenon could be palliated via appropriate design. (For instance, perhaps the fingers could be made retractable, so the dolphin could retract them when it wanted to swim, and extend them when it wanted to manipulate objects.)

This would be a highly experimental adventure in Brain-Computer Interfacing. But, as BCI research advances in the context of human-enhancement applications, I see no reason why it shouldn't advance in the context of dolphin-enhancement applications in parallel.

My thinking is that much of which distinguishes human intelligence from cetacean intelligence is our focus on complex manipulation of tools, and building things (including advanced phenomena like tools that make tools, etc.). If a dolphin brain self-reorganized to adapt to its prosthetic fingers, then the dolphin would have the capability to use tools in a more humanlike way.

Since the cetaceans' evolutionary progenitors had claws of some sort, there may be some vestigial neural wiring in the dolphin brain that will ease the self-reorganization that the dolphin brain needs to go through to make use of the prosthetic fingers.

Another possibility would be to build in the capability for human operators to periodically "take over" the dolphin fingers using remote control. This would serve to show the dolphin what to do with the fingers, both on the conscious reflective level, and on the level of unconscious habituation.

Of course discussions of what to build with the fingers, and how to use tools, could be carried out using Lojban (human or Delphic) ;-D

...

Ahhh ... all the really fascinating research that would get funded if I happened to receive a billion-dollar inheritance from some long-lost uncle ;-p

Why AGI Researchers Should Care about Cetaceans

This is just a brief follow-up to my last post, and a prelude to the one that will follow, which is already brewing in my brain....

In case some of y'all are wondering why I ... whose main intellectual obsession is the creation of AGI systems with general intelligence at the human level and beyond ... have suddenly started ranting about cetacean intelligence, I suppose I should be more explicit about my research-related motivation for digging into the topic

Of course there's a personal motivation -- I love nonhuman animals ... at the moment as well as some humans I share my house with a parrot, 2 dogs and 5 bunnies; and there have been friends of a lot of other species in my life at various times.... In fact the parrot named Abaca might be classified as my best friend over the last few months ;-) ....

I've encountered wild dolphins up close in the shallows of the Indian Ocean when I lived in Western Australia 13-15 years ago, and was certainly struck by the experience, as brief and superficial as it was. I definitely wanted more (and would have sought out more if I hadn't left Western Oz to move to New York and start an AI company)

But what I want to focus on here is my intellectual motivation for, as an AGI researcher, finding cetacean intelligence important.

How often do I hear, among AGI researchers, words to the effect of "Of course we need to model our AGI systems on the human brain and mind, since after all it's the only example we have of a highly generally intelligent system."

I tend to resist this line of thinking ... I think we understand enough about the general mathematics and computer science of cognition that we can understand general intelligence in a manner going beyond the human-specific. My own AI work is an amalgam of aspects directly inspired by human intelligence, and aspects inspired by a broader understanding of intelligence.

But still, there is some point to the common observation that we only know one example of a highly generally intelligent system: the human brain.

But is it actually true?

What if the subset of cetacean intelligence researchers who believe cetaceans have general intelligence comparable to, or greater than, human intelligence ... are actually correct? (Which is my suspicion.)

Then in fact there's another example available -- and we're just not taking the trouble to study it as carefully and thoroughly as we should.

In my immediately previous blog post I gave some links into the cetacean intelligence literature, and some speculations as to what I think the broad nature of cetacean intelligence might possibly be.

In my next blog post I'll discuss some cutting-edge approaches that we might take over the next couple decades to more thoroughly understand cetacean intelligence.

I'm not suggesting that resources be taken from AGI and redirected to cetacean cognitive science: I think that both areas are distressingly underfunded.

In the case that we create AGI programs with superhuman general intelligence before we understand cetacean minds, I think we might still have something to learn from the minds of dolphins. Because cetacean minds may possess a quite different form of intelligence than either us or our AGI creations. And it's hard to tell what may be learned by studying some advanced, fundamentally different incarnation of intelligence.

There could possibly be interesting implications for the study of AGI ethics here, for instance. Cetacea are certainly not optimally ethical creatures ... they're capable of violence just like most other mammals ... but based on what we can understand today, it seems their social organization may have fewer egregious ethical issues than ours. As one example, they seem to have achieved a large-scale, global social organization without warfare. (Evidence of the global nature of their social organization is tentative, but provided by observations such as the way repeated phrases in whale "song" tend to arise in one part of the globe, then spread through whales in all the world's oceans, then die out after a time, replaced by others.)

I'm certainly not suggesting that study of cetacean society will magically provide the answer to the AGI ethics problem, or the problem of generally understanding general intelligence, etc. However, I think it would be very interesting to understand how a fundamentally different sort of general intelligence works, and how it has approached the society/ethics problem, as an additional body of evidence to utilize as we shape the minds of the future.

Cetacean consciousness...

I've been reading many of the writings of John Lilly lately, and also poring through the literature on cetacean intelligence ... and I have to say it's fascinating stuff ....

I'm fascinated by Lilly's cetacean intelligence/communication work, his isolation tank work, even his obsessive (and, apparently, excessive) experiments with ketamine injection leading to long conversations with various hallucinated (?) extraterrestrials ;-)

(I read his stuff a couple decades ago but I've been through a lot of experiences since, and I can read it with different eyes now. I remember how inspirational his book "Programming and Metaprogramming the Human Biocomputer" was for me, when I read it at age 13 or 15 or whatever.)

Anyway ... plenty of scientists by now have followed up Lilly's intuitions about the deep intelligence of dolphins and other cetaceans. A bunch of research papers by various scientists (not under the influence of ketamine ;-) are here:

http://www.dolphin-institute.org/our_research/dolphin_research/dolphinresearchpublications.htm

For some relevant books by people less fringe-y than Lilly, but still quite insightful, see e.g.

• Rachel Smolker's elegant scientific memoir
• This edited volume on Dolphin Societies

but I've found no up-to-date comprehensive review book, so you really gotta read the journal literature and various books to understand what's known so far...

As of now there is no definitive scientific proof that cetaceans are extraordinarily intelligent ... though there's pretty solid proof that they're at least as clever as great apes, I would say (though different in mentality) ...

However, my qualitative impression from reviewing all the evidence is that they are, in some senses, dramatically more intelligent than great apes

I will write something systematic on this topic at some point, when I get more time and have read the literature more thoroughly (obviously this is just a background interest for me, so my reading is going pretty slowly...)

What got me musing about this topic right now was thinking about how the naive physics of our everyday world has impacted human intelligence, and what this might mean for engineering and educating AGI.

Last month Allan Combs and I wrote a paper for the NASA CONTACT workshop, discussing how the radically different environments of extraterrestrials might impact their mind-states and varieties of intelligence:

http://goertzel.org/papers/AlienMinds.pdf

(we'll academic-ize this and publish it somewhere, in time).

And this is also related to a paper I wrote a couple months back, musing about how the lack of fluids, powders, fabrics and other such substances in virtual worlds may impact their utility as homes for humanlike artificial minds:

http://goertzel.org/dynapsyc/2009/BlocksNBeadsWorld.pdf

(In that paper I also explored how it might be possible to enhance virtual worlds to largely remedy this shortcoming, using a special physics-engine technique I called "bead physics".)

In writing that NASA paper, I started wondering how it would impact a mind to evolve in an environment dominated by fluids rather than solids.

My speculation was that, in such a mind, notions of causing and building would be replaced by notions of flowing and shaping .... which would lead to all sorts of other differences.

Gino Yu then pointed out to me these fascinating speculations on the potential subjective experiences of cetaceans:

http://www.cbc.ca/ideas/features/ocean-mind/index.html

All this has spurred me to some of my own entertaining speculations (synthesizing various speculations of Lilly and others) ... to wit:

... what if (as Lilly speculated) the everyday states of mind of cetaceans are more like the states of mind that humans get into while on psychedelic drugs, than they are like our everyday consciousness?

After all, these creatures are breathing deeply and rhythmically ... they're floating in liquid ... generally they're living the sort of physical life that would put humans in a deep semi-meditative state ...

What if their big neocortices are devoted essentially to collaboratively composing and improvising music for each other to listen to?

... but perhaps something more advanced and subtle than human music, reflecting intricate patterns of social interaction, and holistic observations about the state of the underwater ecosystem, and emergences between these social and ecosystem patterns...

This would be a type of intelligence not focused on building tools or solving puzzles in the humanlike sense....

As with human intelligence, the main spur for the evolution of such intelligence would be social. Once the composition/improvisation of this kind of communicative/depictive music became a critical aspect of membership in cetacean society, then there would be evolutionary force to compose/improvise more and more appealing music....

In this hypothesis, the crux of dolphin communication might not be one-to-one conversation, but rather multi-player musical improvisation, with both spatial and temporal aspects. Dyadic conversation with practical import might occur, yet have vastly less complexity and subtlety than other aspects of the musical communication...

One interesting thing about this speculation is that, if it were true, it would mean that probing cetacean intelligence using concepts and methods developed for studying human intelligence, could push the researcher in badly wrong directions.

By analogy, imagine that a species whose main focus of intelligence was collaborative spatiotemporal music improvisation, tried to judge and explore human intelligence. Most humans would be judged as hopelessly moronic ... and then a few gifted musicians might be viewed as moderately intelligent. Due to the other species focusing on collaborative spatiotemporal music improvisation, they would miss what is really the crux of human intelligence: our dyadic linguistic communication, and our tool-building.

John Lilly wanted to probe cetacean communication with computer tech, back in the late 1970s and early 1980s. Computers are a lot better now, so someone could take a much better shot at it. But rather little research seems to be going on at the intersection of advanced AI pattern analysis and cetacean communication, at the moment. Too bad.

More ambitiously, one can envision creating an AI that shared both a humanlike body, and a dolphin-like body, and letting it exist in both worlds.

Lilly did make a good point, that we should probably take some of the $ we are spending on looking for alien lifeforms in space, and devote some of it instead to trying to communicate with these alien intelligences that apparently exist in our oceans. If we can't even communicate with the other intelligences on our own planet, cracking the codes of the minds and languages of beings on alien planets may not be realistic yet (though, of course, there is massive uncertainty in all these domains...).

There is some inordinately silly stuff written about cetacean intelligence -- I read one book on the theme that "Jesus was a dolphin"!! And Lilly certainly complicated his message about cetacean intelligence by mixing it up with some of his other messages, for instance about extraterrestrials whom he felt he contacted while in isolation tanks and on ketamine. But all that is really beside the point. When you look at the scope of existing qualitative evidence about cetacean intelligence, the picture is striking....

Whether the speculations I've made above are on-point or not, I'm convinced there is something very interesting going on in cetacean minds and societies -- which we are not putting nearly enough effort into understanding.

Instead, we are still killing them and making them into steaks.

Female mad scientists and creative nihilism (thoughts on Sofia Kovalevskaya)

I've been thinking of making the protagonist of my next novel a female "mad scientist", but I wasn't sure how to write the character, so I started searching history for a good model.

I found damn few female mad scientists in recorded history.

The closest I found was the Russian mathematician Sofia Kovaleveskaya, whose name I knew from the Cauchy-Kovalevsky Theorem in partial differential equations. I hadn't known much about her before, so I did some reading and found she fit the bill pretty well:

• World-class mathematician
• Spent some time inventing weird new electrical machinery
• Accomplished novelist
• Also wrote plays and poetry
• Participated in the Paris Commune, and generally schemed for revolutionary overthrow of governments
• Helped her husband lose piles of money during a several year period devoted to "clever" real estate and financial speculation

A woman after my own heart -- wish I'd known her! She was also the first woman ever to get a math PhD (she lived in the mid-1800's).

Her biography is also full of nice tidbits, like

• She first got passionate about advanced math when her attic happened to get wallpapered with lecture notes from a calculus class her father had taken years before. So she learned about limits and such from reading unordered pages of mathematical text pasted to a wall!
  
• To study advanced math, she had to leave Russia (due to sexist regulations), and to do that she had to get married ... so she entered into a "fake marriage" with a platonic male friend with the sole purpose of escaping Russia to get to university in Western Europe (although, many years later, the fake marriage turned real...)
  

Her childhood memoir "A Russian Childhood" is a wonderful book and I'd recommend it to anyone who likes Russian literature.

This biography is also worth reading for the story it tells -- although the biographer's radical-feminist antimasculism is annoying, and appears to radically falsify Kovalevskaya's relationship with her husband, among other things

There's also a historical novel about her life, which I haven't read

Her novella "Nihilist Girl" has some greatness about it too, but feels first-draft-ish, like it needed a final edit to really become a work of art.

But the main thing I wanted to write about today was the revised idea of "nihilism" I got from reading "Nihilist Girl" and these other materials....

I've generally thought of "nihilism" as meaning "believing in nothing" ... or at least the attitude of Turgenev's character Bazarov, that nothing really matters much including one's own life....

But after reading Kovalevskaya, I realize that -- in thinking about Russian nihilism from the mid-1800s -- I was largely mistaking the parody for the real thing.

Kovalevskaya's brand of nihilism was significantly more interesting than that of the fictional character Bazarov.

It wasn't about absolutely rejecting everything and judging everything as meaningless and worthless. Rather, it was about rejecting any absolute values. It was about rejecting anything as sacred -- and opening everything up to question.

What was being rejected was a world-view in which there are certain absolute truths, in terms of which everything else must be assessed.

If you get rid of absolute truth, though, then what are you left with? Complete worthlessness and suicide, a la Bazarov? Or maybe not. What Kovalevskaya and her friends were after was something different.

One can think about it in terms of self-organization and strange attractors. Once one gets rid of absolute truth, and admits every single thing as open to question and revision, then one has a self-organizing dynamical system in which each thing gets potentially revised by each other thing. But the outcome of this doesn't need to be a homogeneous evaluation of everything as worthless. The outcome can be some other "strange attractor" in which each thing gets value from each other thing, according to a complex system of interdependencies and interactions.

Kovalevskaya-style nihilism, it seems, wasn't really about rejecting everything as equally worthless, but more about rejecting anything as absolutely valuable ... and letting the process of interactive, adaptive, mutual-value-adjustment spread through everything and lead to a productive evolution of new valuations and forms.

But I don't seem to have been the only one to get confused about nihilism.

Wikipedia says:

Nihilism (from the Latin nihil, nothing) is the philosophical position that values do not exist but rather are falsely invented. Most commonly, nihilism is presented in the form of existential nihilism which argues that life is without meaning, purpose or intrinsic value. Moral nihilists assert that morality does not exist, and subsequently there are no moral values with which to uphold a rule or to logically prefer one action over another.... The term nihilism is sometimes used synonymously with anomie to denote the general mood of despair at the pointlessness of existence that one has when they realize there are no necessary norms, rules, or laws.

and Nietzsche wrote in his notebooks (The Will to Power, section 585, translated by Walter Kaufmann)

A nihilist is a man who judges of the world as it is that it ought NOT to be, and of the world as it ought to be that it does not exist. According to this view, our existence (action, suffering, willing, feeling) has no meaning: the pathos of 'in vain' is the nihilists' pathos — at the same time, as pathos, an inconsistency on the part of the nihilists.

Nietzsche posited his own views as dramatically contradictory to nihilism -- and they certainly are wholly contradictory to Bazarov-style nihilism ... Nietzsche was all about creating your own values, rather than accepting any values as absolute, and rather than rejecting all values.

But it seems that Nietzsche was posing nihilism as a "straw man" to an even greater extent than I'd thought before ... and that his general views on trans-nihilist value-creation were not so fundamentally different than those of many of the Russian nihilists of the 1860s.

In Cities of the Red Night, Burroughs wrote "Nothing is true; everything is permitted" -- which on one reading reflects Bazarov-style nihilism ... and which Burroughs borrowed from Nietzsche, whose Zarathustra said:


“Nothing is true, all is permitted”: so said I to myself. Into the coldest water did I plunge with head and heart. Ah, how oft did I stand there naked on that account, like a red crab! –

But if one reads this Burroughs/Nietzsche aphorism as "Nothing is absolutely true; nothing is absolutely impermissible" then one has a Kovalevskaya-style nihilism, in which dogmatism is eliminated in favor of the creative self-organization of new value systems.

Dostoevsky also seems to have put a lot of energy into pillorying a straw-man version of Russian nihilism. Dostoevsky's nihilists are folks like Raskolnikov (the heartless, utilitarian murderer of Crime and Punishment) or Kirilov in The Possessed (a majorly hilarious character who preaches copiously about the worthlessness of existence and then suicides because he considers it the highest act of free will).

Kovalevskaya's "nihilist girl" character is about as far as you can get from Raskolnikov -- an extremely caring person, she marries a political prisoner (a man twice her age in whom she has no romantic interest) to save him from near-certain execution, even though this means her own exile to Siberia ... and generally decides to devote her life to helping Siberian prisoners, as a way of contributing to the common good. She doesn't lack values -- she just rejects having absolute values imposed on her, and wishes to create her own values based on her own intuitions and her engagement with the world.

And the narrator of Nihilist Girl makes a different choice -- she is a mathematician like Kovalevskaya, devoted to the life of science; she plainly states that she would not exile herself to save a political prisoner -- yet she just as plainly shares the same underlying philosophy of "creative nihilism" (my phrase, not Kovalevskaya's).

Dostoevsky courted Sofia Kovalevskaya's big sister Aniuta, as it happened. His story was that he broke off their engagement because she was too nihilistic. Her story (which has more ring of truth) was that she broke up with him, before they were formally engaged, because she didn't want to spend her life taking care of him and wanted more freedom to explore her own interests and passions.

But however the soap opera really went down, both Sofia and Aniuta were too nihilistic for Dostoevsky. The latter believed that only religious belief could save you from destructive nihilism of the form demonstrated by Bazarov, Raskolnikov or Kirilov. He didn't think new value systems could self-organize out of a pool of interacting non-absolutes ... to him value needed to begin with some absolute faith, some absolute assumptions.

All in all, I conclude that nihilism suffered from poor marketing, and an overly subtle and ironic name, which caused its more interesting variants to get forgotten, and its less interesting variants to get repeatedly parodied.

Long live creative nihilism, Kovalevskaya style!

Nothing is true; everything is permitted ;-)

When the Net Becomes Conscious

A journalist emailed me today and asked me some questions about the possibility of the Internet becoming conscious. The questions and my answers follow:


> 1) Why do some people think it is possible for the internet (or internet plus humans) to become conscious? Is it to do with the network architecture?

Many scientists believe that consciousness is a property that will inevitably emerge from any complex system that has the right sort of internal dynamics, and the right sort of interaction with its environment.

Exactly what the right sort of dynamics and interactions are, different theorists disagree on. But it seems plausible that the Internet may have enough of them to develop its own sort of consciousness. The Internet perceives and acts on the world; it stores declarative, episodic and procedural memories; it recalls some information and forgets others; etc. In short it behaves a fair bit like a human mind, though there are a lot of differences too.

According to this perspective, the Internet might already have a degree of consciousness, though of a type quite different from human consciousness.

Neuroscientist Susan Greenfield views consciousness as consisting of "whole-brain activation patterns". In this sense one would say that the Internet of today has a more fragmented, dissociated consciousness than a human mind ... there aren't so many "whole-internet activation patterns", though there are intense patterns spanning large portions of the Internet.

Of course, there are many philosophies of consciousness. My own view of consciousness is a bit eccentric for the scientific world though rather commonplace among Buddhists (which I'm not): I think consciousness is everywhere, but that it manifests itself differently, and to different degrees, in different entities.

So to me the interesting question is whether the Internet has (or will develop) consciousness of the same type as humans, or maybe even of a more advanced and intricate type.

It seems that as the Internet expands and grows richer, it *could* develop a more human-like, more unified consciousness than it has now ... with more coherent "whole Internet activation patterns"...

> 2) What might be the consequences of such an event? Do you think it might be something that we should welcome?

The potential consequences of the Internet developing more coherent holistic activation patterns (ergo more humanlike consciousness) are rather difficult to predict, I find!

However, I personally am pessimistic about the future in the case that humans remain the most powerful minds on the planet. I don't trust us to use our increasingly advanced technologies in an ethical and nondestructive way.

So I think the outlook for humanity is probably better in the case that an emergent, coherent and purposeful Internet mind develops, than in the case where it doesn't.

But there is a lot of uncertainty in either case!


> 3) If it were possible, what would be needed to make the internet conscious? How far away from that situation are we?

My guess is that humanlike consciousness is not going to spontaneously evolve from the Net. However, I think someone could engineer it, by specifically creating an AI system on a server farm, oriented toward serving as a kind of "central cognition engine" for the Internet as a whole.

This central cognition engine wouldn't need to control everything on the Net; it would just need to read a lot of the information out there on the Net, and then insert information of its own creation in appropriate locations (posting to email lists, creating web pages, buying and selling things, etc.).

The engine might be created with some other primary purpose (e.g. as an artificial scientist aimed at making new discoveries via collaborating with human scientists online), or it might be created specifically with the goal of transforming the Internet into a more coherent, more humanlike intelligence. Either way the effect might be the same.

This is the scenario I described in my 2001 book "Creating Internet Intelligence," and I still think it is a plausible one.

Once Science Eliminates Pain ...

The bored or overly curious may check out my latest neurological dysfunction (aka work of fiction) "The Last Aphrodisiac", at

http://goertzel.org/fiction.htm#pain


What happened was, I was driving late at night listening to a Morphine CD in the car, then got home, lay in bed and fell asleep with the song "Cure for Pain" in my head.

I had a number of dreams on the theme (what if pain were really eliminated, in some interesting sense? what would life be like? what if it were rediscovered?) and woke up plagued by this story. On a cross-country flight to a weekend workshop on "Evaluation and Metrics for Human-level AI", I decided to write it down...

At first I thought it would take a single page to write down, but it wound up 15 pages, and the punchline doesn't start to unfold till page 7 or 8.

This is the first story I've written in a long time that doesn't involve AI in any serious way. Rather, it uses future tech like uploading-to-superhuman-form and cranial jacks to enlarge upon certain aspects of human relationships, especially romantic ones. It's probably the closest thing to a maudlin love story I'll ever write (well, I hope so).

Ahh, the things that can transpire between a man, a woman, and an illicit cranial jack modification device... ;-)

Freaky Russian Paranormal Biology (plus irrelevant personal-history rambling)

A friend pointed me to this paper

http://www.emergentmind.org/gariaev06.htm

which I found provoking and interesting, though nowhere near fully convincing....

The authors argue that much "junk DNA" actually serves as an interface to some sort of energy-informational field reminiscent of Sheldrake's "morphogenetic field"...

Their comments on traditional biology are a bit naive: in the last 5 years a lot of functions of formerly-known-as-junk-DNA have been found. So it's no longer true that ordinary biology says 95%+ of DNA is useless. We're finding this DNA serves a lot of regulatory functions, even though it doesn't code for proteins.

On the other hand, that oversight doesn't make their theory wrong; it just makes them out of date regarding traditional biology.

The experimental results they describe are certainly compelling and intriguing. Fraud is always a possibility, yet, I'm wary to dismiss results as likely fraud just because they violate currently standard scientific theories. (After all, every scientific paradigm prior to the current ones has been overthrown, right?)

As an aside in their discussion, they mention some old results involving Kirlian photography. I tried to build a Kirlian camera in my basement once, in Randolph NJ in the late 1990s, with a view toward trying to replicate the "phantom leaf effect" (see link below) ... but wound up setting fire to part of the basement instead. (I did build a Tesla coil from a neon sign transformer, and it worked a few times, but eventually it caught fire before I finished making it into a Kirlian camera apparatus.)

A recent attempt to replicate the phantom leaf effect, with intriguing results, is described here:

http://shadowboxent.brinkster.net/lemurkirlian2.html

It's hard to be anywhere near certain, but I'm intuitively inclined to feel there might be some truth somewhere in the vicinity of these guys' wacky theory.

Obviously it's close to my own "glocal theory of psi" ... in the language of my "glocal memory theory" what they're saying is that living organisms have "keys" (the observed physical substance) and "maps" (the correlated energy-information field).

I'm hesitant to use the word "energy" in this context as these Russian authors do, because "energy" has a specific meaning in physics, and this (even if real) may be something different. I have thought of their energy-informational field as a kind of "pattern space", more like Sheldrake's idea of a morphogenetic field....

But, without some hypothesis regarding the dynamical laws of this posited morphogenetic field, their theory remains more philosophical than even "speculatively scientific."

It seems that conceivably -- if there's any reality here -- one could learn something about the laws of this field via systematically varying parameters in Kirlian photography experiments.

Yet another line of research I'd probably fund if I were super-wealthy ;-p

P.S.

(warning: loosely related personal-history rambling below...)

Wow, I remember now that when I was 18 years old and in my senior year of college (at Simon's Rock, in western Massachusetts) and visited CalTech, where I was hoping/considering to go to grad school, I mentioned to one of the math profs there that I was interested in doing a PhD thesis on using partial differential equations to model bioelectromagnetic fields as had been discovered in some strange Russian experiments.

His reply was something like "Well, you certainly know a lot of big words, but do you know any math?" We talked a bit and he discovered that I did; but I didn't get admitted to CalTech anyway.... It was obvious that the math department there did not like that potential thesis topic!!

On that same visit to California I visited Berkeley's Logic and Methodology PhD program, where I expressed my interest in writing a thesis on using hypersets (non-well-founded sets) to model consciousness (which now is one of the themes of one of the handful of half-finished books on my hard drive). This was better received, and they were likely to admit me with funding (or so they said verbally) but I wound up not completing my application because the students there told me that the department inevitably made its students take 7-9 years to finish their PhDs. I didn't want to be in school that long, so I wound up at NYU's Courant Institute instead, which I liked because it combined math, theoretical physics and computer science in one department...

On that trip I also visited a girl at UCSD whom I had a crush on (from when she'd attended college with me in Massachusetts), and was disturbed to find she'd become a fundamentalist Christian, handing out Jesus brochures on the street. (I saw her 11 years ago and she seemed to have gotten over that phase long ago, fortunately... though it was a bad visit as I'd been up nearly the entire night, insomniac due to too much arguing with my wife-at-the-time Gwen, and was completely bleary-eyed and -minded for the whole visit ... that was a few days before the birth of my daughter Scheherazade and Gwen was mad at me for repeatedly getting stoned out of my mind with one of my friends at a time when the baby might pop out at any minute ... well, at least being totally stoned helped the fighting go down easier!!)

In hindsight, that was a rather entrepreneurial trip for a college senior to make (esp. an 18 year old one): I basically invited myself on interviews to grad schools I was interested in, prior to even completing my formal applications to the places ... to go talk to the profs and students there, sit in on classes, and get a feel for the departments. I didn't realize at the time that this was a fairly eccentric thing to do. But it was a good idea.... Although I had only one set of clothes for the whole trip, because People Express Airlines (which featured a great $99 cross country flight) sent my luggage to Europe by mistake.

(Unfortunately that trip caused me to get fired from my job as a math homework grader, because I just took off from college for a week and flew to California without giving any notice to my boss, so all the papers went ungraded while I was gone. I wasn't too conscientious back then. I'm still a slob with paying bills for the house and such, but I try not to be a mess like that in professional life anymore! Still it's hard to focus on reality and not be an absent-minded professor sometimes ;-)

All that probably has something to do with Kirlian photography, auras and Russian theories of morphogenetic fields ... but since I don't get stoned hardly ever these days, I'm not in the right state of mind to find the connecting thread; and I'll go cook dinner instead and then get down to some useful work (oh yeah, and Scheherazade and I are going to watch the film Baghead tonight... which unfortunately has nothing to do with Buckethead...)

Hmmm... that reminds me of the Timothy Leary Family Reunion I attended in San Francisco last week ... what a wonderful assortment of old hippies with cosmic looks in their eyes and wild ideas in their brains; along with various sympathetically resonating youngsters like me ... but, I won't go there right now...

Nietzschean Nonlinear Economics?

I was thinking a little more about the current economic situation and why it seems so confusing to people, including experts.

Part of the problem is just that the systems involved are so complex, of course -- the economy has complexified and the human brain has not kept up.

Another part of the problem, though, is that none of the traditional economic theories (neither the mathematical ones or the qualitative ones) embrace this complexity in principle.

I.e., the very fact of the increased complexity of the economy has direct implications, which seem to require introduction of new approaches for economic analysis.

One way to look at this is in terms of what I semi-seriously call "Nietzschean economics," where a more abstract notion of economic power replaces the traditional notion of money.

Put simply: it seems that the modern economy has somewhat obsoleted the concept of money, which has economists (and many others) confused...

For instance, China has a lot of US dollars right now, in principle ... but they can't really spend most of it, because if they traded their dollars for goods on the open international market, then the dollar would collapse in the international currency markets, the US economy would tank, demand for Chinese goods would collapse, and China would risk massive internal unrest etc.

It's not exactly that the Chinese government has money they can't use; but they have money that comes with severe restrictions on the ways it can be used. Having this money confers great economic power on them; but, this economic power can't necessarily be used in the traditional manner of buying stuff.

In general: with the world so interconnected, the notion of a unit of currency as being something that can be exchanged for a certain amount of stuff, anywhere, doesn't really apply...

More complexly, the same sorta phenomenon applies among and within investment banks and other financial institutions. E.g. a large bank holds some portfolio of financial instruments ... but if they sell a lot of one kind of instrument, this impacts the markets in a way that affects the value of the others, etc. So one can hardly assign each instrument they hold a value independently of the others.... The values of their various portfolio items are interdependent in the manner of a system of simultaneous nonlinear equations.

What this growing, rampant interdependence means is that the traditional concepts and tools of economics don't closely apply to the international biz/finance world anymore...

A lot of this was foreseen, on a qualitative level, in Galbraith's book "Economics and the Public Purpose" from the 1970s ... he talked about the "market economy" versus the "technostructure" and pointed out that the latter (being a complex of large corporations, governments and other institutions) follows quite different rules.

Well, now the nonlinear dynamics of the technostructure is running rampant -- but economists are still mainly studying it with tools designed for studying markets.

One step I think that needs to be taken, on the theory level, is to view actual buying power as Level 1 of a hierarchy of types of economic power. I'm thinking of:

• Level 1 = power to buy goods or services [right now, or at some future point(s) in time]
• Level 2 = power to influence others' Level 1 power
• Level 3 = power to influence others' Level 2 power
• .. etc. ...

In principle, one could boil down Level k power into Level 1 power. But in practice, with the economy so complex, this involves calculations that are infeasible to do. So, economic agents are in effect seeking Level k power without a clear picture of how it will in the future boil down into Level 1 power. This might be thought of as Nietzschean economics (as Nietzsche viewed the "will to power" as the essential dynamic of the universe).

The ultimate extreme of all this of course would be

• Level infinity = power to influence others' Level infinity power

which was basically Nietzsche's view of the driving force of the universe ... and I do think that international economics and politics boils down to this sometimes: power for it's own sake, rather than being tied to ultimately influencing the acquisition of goods and services.

Traditional economics is based on the notion that everything boils down to buying power ... but in a world where no one is smart enough to calculate what their decisions will imply in terms of buying power, this sort of economics seems to have limited applicability...

What we need is a nice, elegant, pragmatically applicable theory of the nonlinear dynamics of Level k economic power under conditions where

• the computational complexity of recognizing important high-level patterns in an economy

vastly exceeds

• the computational capability of even the smartest individual participants in the economy

This might be a fun thing to work on, but I've got a thinking machine to build, so hopefully somebody else will do it ... or we'll have to wait for the AI to solve the problem. (Of course, an appropriately constructed AI could also palliate the problem, due to having increased capability to recognize economic patterns, either due to possessing greater-than-human general intelligence, or due to combining human-level general intelligence with specialized capabilities for economic analysis.)

The Subtle Structure of the Everyday Physical World = The Weakness of Abstract Definitions of Intelligence

In my 1993 book "The Structure of Intelligence" (SOI), I presented a formal definition of intelligence as "the ability to achieve complex goals in complex environments." I then argued (among other things) that pattern recognition is the key to achieving intelligence, due to the algorithm

• Recognize patterns regarding which actions will achieve which goals in which situations
• Choose a goal that is expected to be good at goal achievement in the current situation

The subtle question in this kind of definition is: How do you average over the space of goals and environments? If you average over all possible goals and environments, weighting each one by their complexity perhaps (so that success with simple goals/environments is rated higher), then you have a definition of "how generally intelligent a system is," where general intelligence is defined in an extremely mathematically inclusive way.

The line of thinking I undertook in SOI was basically a reformulation in terms of "pattern theory" of ideas regarding algorithmic information and intelligence that originated with Ray Solmonoff; and Solomonoff's ideas have more recently been developed by Shane Legg and Marcus Hutter into a highly rigorous mathematical definition of intelligence.

I find this kind of theory fascinating, and I'm pleased that Legg and Hutter have done a more thorough job than I did of making a fully formalized theory of this nature.

However, I've also come to the conclusion that this sort of approach, without dramatic additions and emendations, just can't be very useful for understanding practical human or artificial intelligence.

What is Everyday-World General Intelligence About?

Let's define the "everyday world" as the portion of the physical world that humans can directly perceive and interact with -- this is meant to exclude things like quantum tunneling and plasma dynamics in the centers of stars, etc. (though I'll also discuss how to extend my arguments to these things).

I don't think everyday-world general intelligence is mainly about being able to recognize totally general patterns in totally general datasets (for instance, patterns among totally general goals and environments). I suspect that the best approach to this sort of totally general pattern recognition problem is ultimately going to be some variant of "exhaustive search through the space of all possible patterns" ... meaning that approaching this sort of "truly general intelligence" is not really going to be a useful way to design an everyday-world AGI or a significant components of one. (Hutter's AIXItl and Schmidhuber's Godel Machine are examples of exhaustive search based AGI methods.)

Put differently, I suspect that all the AGI systems and subcomponents one can really build are SO BAD at solving this general problem, that it's better to characterize AGI systems

• NOT in terms of how well they do at this general problem

but rather

• in terms of what classes of goals/environments they are REALLY GOOD at recognizing patterns in

I think the environments existing in the everyday physical and social world that humans inhabit are drawn from a pretty specific probability distribution (compared to say, the "universal prior," a standard probability distribution that assigns higher probability to entities describable using shorter programs), and that for this reason, looking at problems of compression or pattern recognition across general goal/environment spaces without everyday-world-oriented biases, is not going to lead to everyday-world AGI.

The important parts of everyday-world AGI design are the ones that (directly or indirectly) reflect the specific distribution of problems that the everyday world presents an AGI system.

And this distribution is really hard to encapsulate in a set of mathematical test functions. Because, we don't know what this distribution is.

And this is why I feel we should be working on AGI systems that interact with the real everyday physical and social world, or the most accurate simulations of it we can build.

One could formulate this "everyday world" distribution, in principle, by taking the universal prior and conditioning it on a huge amount of real-world data. However, I suspect that simple, artificial exercises like conditioning distributions on text or photo databases don't come close to capturing the richness of statistical structure in the everyday world.

So, my contention is that

• the everyday world possesses a lot of special structure
• the human mind is structured to preferentially recognize pattern related to this special structure
• AGIs, to be successful in the everyday world, should be specially structured in this sort of way too

To encompass this everyday-world bias (or other similar biases) into the abstract mathematical theory of intelligence, we might say that intelligence relative to goal/environment class C is "the ability to achieve complex goals (in C) in complex environments (in C)"

And we could formalize this by weighting each goal or environment by a product of

• its simplicity (e.g. measured by program length)
• its membership in C, considering C as a fuzzy etc

One can create a formalization of this idea using Legg and Hutter's approach to defining intelligence also.

One can then characterize a system's intelligence in terms of which goal/environment sets C it is reasonably intelligent for.

OK, this does tell you something.

And, it comes vaguely close to Pei Wang's definition of intelligence as "adaptation to the environment."

But, the point that really strikes me lately is how much of human intelligence has to do, not with this general definition of intelligence, but with the subtle abstract particulars of the C that real human intelligences deal with (which equals the everyday world).

Examples of the Properties of the Everyday World That Help Structure Intelligence

The propensity to search for hierarchical patterns is one huge example of this. The fact that searching for hierarchical patterns works so well, in so many everyday-world contexts, is most likely because of the particular structure of the everyday world -- it's not something that would be true across all possible environments (even if one weights the space of possible environments using program-length according to some standard computational model).

Taking it a step further, in my 1993 book The Evolving Mind I identified a structure called the "dual network", which consists of superposed hierarchical and heterarchical networks: basically a hierarchy in which the distance between two nodes in the hierarchy is correlated with the distance between the nodes in some metric space.

Another high level property of the everyday world may be that dual network structures are prevalent. This would imply that minds biased to represent the world in terms of dual network structure are likely to be intelligent with respect to the everyday world.

The extreme commonality of symmetry groups in the (everyday and otherwise) physical world is another example: they occur so often that minds oriented toward recognizing patterns involving symmetry groups are likely to be intelligent with respect to the real world.

I suggest that the number of properties of the everyday world of this nature is huge ... and that the essence of everyday-world intelligence lies in the list of these abstract properties, which must be embedded implicitly or explicitly in the structure of a natural or artificial intelligence for that system to have everyday-world intelligence.

Apart from these particular yet abstract properties of the everyday world, intelligence is just about "finding patterns in which actions tend to achieve which goals in which situations" ... but, this simple meta-algorithm is well less than 1% of what it takes to make a mind.

You might say that a sufficiently generally intelligent system should be able to infer these general properties from looking at data about the everyday world. Sure. But I suggest that would require a massively greater amount of processing power than an AGI that embodies and hence automatically utilizes these principles? It may be that the problem of inferring these properties is so hard as to require a wildly infeasible AIXItl / Godel Machine type system.

Important Open Questions

A couple important questions raised by the above:

1. What is a reasonably complete inventory of the highly-intelligence-relevant subtle patterns/biases in the everyday world?
2. How different are the intelligence-relevant subtle patterns in the everyday world, versus the broader physical world (the quantum microworld, for example)?
3. How accurate a simulation of the everyday world do we need to have, to embody most of the subtle patterns that lie at the core of to everyday-world intelligence?
4. Can we create practical progressions of simulations of the everyday world, such that the first (and more crude) simulations are very useful to early attempts at teaching proto-AGIs, and the development of progressively more sophisticated simulations roughly tracks the development of progress in AGI design and development.

The second question relates to an issue I raised in a section of The Hidden Pattern, regarding the possibility of quantum minds -- minds whose internal structures and biases are adapted to the quantum microworld rather than to the everyday human physical world. My suspicion is that such minds will be quite different in nature, to the point that they will have fundamentally different mind-architectures -- but there will also likely be some important and fascinating points of overlap.

The third and fourth questions are ones I plan to explore in an upcoming paper, an expansion of the AGI-09 conference paper I wrote on AGI Preschool. An AGI Preschool as I define it there is a virtual world defining a preschool environment, with a variety of activities for young AI's to partake in. The main open question in AGI Preschool design at present is: How much detail does the virtual world need to have, to support early childhood learning in a sufficiently robust way? In other words, how much detail is needed so that the AGI Preschool will posssess the subtle structures and biases corresponding to everyday-world AGI? My AGI-09 conference paper didn't really dig into this question due to length limitations, but I plan to address this in a follow-up, expanded version.

The Increasing Value of Peculiar Intelligence

One more sociopolitical/futurist observation ...

If you're not aware of David Brin's Transparent Society concept, you should read the book, and start with the Web page

http://www.davidbrin.com/tschp1.html

His basic idea is that, as surveillance technology improves, there are two possibilities:

• The government and allied powers watch everybody, asymmetrically
• Everybody watches everyone else, symmetrically (including the government and allied powers being watched)

He calls the latter possibility sousveillance ... all-watching ...

What occurs to me is that in a transparent society, there is massive economic value attached to peculiar intelligence

This is because, if everyone can see everything else, the best way to gain advantage is to have something that nobody can understand even if they see it

And it's quite possible that, even if they know that's your explicit strategy, others can't really do anything to thwart it...

Yes, a transparent society could decide to outlaw inscrutability. But this would have terrible consequences, because nearly all radical advances are initially inscrutable....

Inscrutability is dangerous. But it's also, almost by definition, the only path to radical growth.

I argued in a recent blog post that part of the cause of the recent financial crisis is the development of financial instruments so complex that they are inscrutable to nearly everyone -- so that even if banks play by the rules and operate transparently, they can still trick shareholders (and journalists) because these people can't understand what they see!

But it seems that this recent issue with banks is just a preliminary glimmering of what's to come....

Yes, maybe there is something a bit self-centered or self-serving in this theory, since I seem to personally have more differential in "peculiar intelligence" than in most other qualities ... but, call it what you will, my peculiar intelligence is definitely pushing me to the conclusion that peculiar intelligence is going to be a more and more precious commodity as the Age of Transparency unfolds...

Obviously, you can also see this phenomenon in financial trading even in a non-crisis time. It's not enough to be smart at predicting the markets to make a LOT of money ... because if you're just smart, but in non-peculiar way, then once you start trading a lot of money, others will observe your trades and pick up the pattern, and start being smart in the same way as you ... and then you'll lose your edge. The way to REALLY make a lot of money in the markets is to be very smart, and in a very peculiar way, so that even if others watch your trades carefully, they can't understand the pattern, and they can't imitate your methodology....

The best trader I know personally is Jaffray Woodriff who runs quantitative.com, and he exemplifies this principle wonderfully: very intelligent, very peculiar (though in an extremely personable, enjoyable way), and very "peculiarly intelligent" ;-)

Democratic market socialism redux

All in all, the conclusion I'm coming to lately ... as reflected in my last two blog posts, as well as in some other thinking ... is that government is going to need to do some rather careful and specific things to guide society toward a positive Singularity.

Yes, if someone creates a hard-takeoff with a superhuman AGI, then the government and other human institutions may be largely irrelevant.

But if there is a soft takeoff lasting say 1-3 decades ... before the hard takeoff comes along ... then, my view is increasingly that the market system is going to screw things up, and lead to a situation where there are a lot of unhappy and disaffected people ... which increases the odds of some kind of nasty terrorist act intervening and preventing a positive Singularity from occurring.

It seems we may need to review the general line of thinking (though not many of the specific proposals) from old democratic-socialism-style texts like Economics and the Public Purpose ...

Perhaps one positive consequence of the current economic crisis is that it may cause the US public to understand the value of well-directed government spending....

And yes, I'm well aware that most of my colleagues in the futurist community tend to be libertarian politically. I think they're just wrong. I am all in favor of getting rid of victimless crimes ... legalizing drugs and prostitution and so forth ... but, given the realities of the next century and the risks of a negative Singularity, I don't think we can afford to leave things up to the unpredictable self-organizing dynamics of the market economy ... I think we as a society will need to reflect on what we want the path to Singularity to be, and take specific concerted governmental actions to push ourselves along that path...

This is not the political dialogue for 2008 ... different issues are occupying peoples' minds right now ... but it may be the political dialogue for 2012 or 2015 ... or at latest, I'd guess, 2020 ...

Why the average workweek isn't decreasing faster ... and what we can do about it

This is another post on political, economic and futurist themes ... starting out with a reflection on a bogus patent, and winding up with a radical social policy proposal that just might improve life in the near future and also help pave the way to a positive Singularity... (yeah, I know I know, lack of ambition has never been one of my numerous faults ;-)

Let's start with the simple, oft-asked question: Why isn't the average workweek decreasing faster ... given all the amazing technology recently developed?

One clue is found in some news I read today: IBM has patented the idea of a specialized electronic device that makes it handier to split your restaurant bill among several people at the table:

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=7,457,767.PN.&OS=PN/7,457,767&RS=PN/7,457,767

The patent application is really quite hilarious reading ... and the number of references to prior, equally bullshitty patents for related inventions is funny too.

What strikes me most here is the amount of effort expended by lawyers, patent examiners and judges in dealing with this crap. Not to mention their paralegals, secretaries, and on and on and on.

Why does this part of the contemporary human workload exist?

One could argue that some wasted work is a necessary part of a complex economic system, and that it would be hard to eliminate the crap without throwing out a lot of useful stuff as well.

I'm sure this is part of the truth, but I don't think it's the whole story.

Another part of the answer, I think, is: This kind of BS work exists because people have time to do it.

If people didn't have time to do this shit, because they all had to be occupied gathering food or making shelter or defending themselves against attackers -- or conceiving or manufacturing truly original and interesting inventions -- then the legal system would rapidly get adjusted so as to make bullshit patents like this illegal.

But, because we have plenty of extra resources in our economy ... due to the bounty created by advances in technology ... we (collectively and without explicitly intending to) adjust various parameters of our economy (such as the patent system) in such a way as to create loads of bullshit work for ourselves.

But there's a little more to it than this. Granted that we have the time ... but why do we choose to spend our time on this kind of crap? Instead of, say, doing less useless work and having more free time?

One important, relevant fact to grasp is that people like working.

Yes, it's not just that we're ambitious or greedy, it's that we actually like working -- in spite of the fact that we also like complaining about working. (Don't blame me -- humans are just fucked-up creatures....).

There is loads of data supporting this idea; I'll cite just a smattering here.

Americans work more and are happier than Europeans:

http://sayanythingblog.com/entry/americans_work_harder_happier_than_europeans/

Old folks who work longer are happier:

http://ideas.repec.org/p/pra/mprapa/5606.html

I can't remember the link, but studies show that people are on average happier at work than at home (if asked at random times during the day), but will when asked SAY they are happier at home than at work.... I think this study was done by Mihaly Csikszentmihalyi, who you can find out about at

http://www.time.com/time/health/article/0,8599,1606395,00.html

http://www.amazon.co.uk/Flow-Classic-Work-Achieve-Happiness/dp/0712657592

He is the pioneer of the study of "flow", the state of supreme happiness that many humans enter into when carrying out some highly absorbing, satisfying activity, like creating an artwork, participating in a sport, acting in a play or doing scientific or other professional work ... or gardening ... or chopping down a tree ... actually the activity can be almost anything, the real point is in the way the mind interacts with the activity: it has to be enough to fully occupy the mind without causing too much frustration, so that the self is continually lost and rediscovered within the ongoing dynamism of the interactive process....

Among many more interesting observations, he notes that:

"
Those at work generally report that they wish they were at home, but when they're home they often feel passive, depressed or bored. "They have in mind that free time at home will make them feel better, but often it doesn't,"
"

But, the plot thickens ... because, although we like working ... this doesn't necessarily mean we like working long hours.

People who work part-time are happier than those working full-time (84 per cent are happy versus 79 per cent):

http://www.arboraglobal.com/documents/Happiness%20at%20Work%20Index%202007.pdf

So where we seem to fail is in creating more part-time jobs...

This seems to be because, for an employer, it's always more cost efficient to hire one full time worker than two part-timers ;-(

So, instead of making use of the bounty provided by technology by creating part-time jobs and also creating more opportunities for creative, growth-producing, flow-inducing leisure ... we make use of it by creating more and more full-time jobs doing more and more bullshit work, like processing patents for "inventions" like the one I referenced above.

Given the way our brains work, we're better off working full-time than we would be not working at all.

But there is at least some evidence to suggest we'd be even better off working fewer hours....

But, given the way the market system works, there is almost never any incentive for any employer to allow part-time workers. It just isn't going to be rational from their perspective, as it will decrease their economic efficiency relative to competitors.

The market system, it seems, is going to push toward the endless creation of BS work, because no individual company cares about reducing the workweek ... whereas individual companies DO in many cases profit by creating BS work ... if they can bill other companies (or the government) for this bullshit work....

So, this leads to the idea that the government of France may have had the right idea at heart, in creating the 35 hour maximum workweek. Which they have since rescinded, because in a global context, it made them uncompetitive with other countries lacking such a restriction.

But anyway, outlawing long work hours is obviously not the answer. People should have the freedom to work long hours and get paid for them, if they want to.

So, an interesting question, policy-wise, is how the government can create incentives for reduced workweeks, without introducing punitive and efficiency-killing regulations.

One possibility would be, instead of just government projects aimed at paying people to build infrastructure, to launch government projects aimed at paying people to do interesting, creative, growth-inducing stuff in their spare time.

Basically, what I'm suggesting is: massively boosted government funding for art and science and other not-directly-economically-productive creative work ... but without making people jump through hoops of fire to get it (i.e. no long, tedious, nearly-sure-to-fail grant applications) ... and specifically designed NOT to discriminate against people who do not-directly-economically-productive creative work only part-time.

This would make it harder for companies to find full-time employees, because it wouldn't be all that much more lucrative to work full-time than to work part-time plus earn a creative-activity stipend on the side. But, unlike France's previous restrictive laws, it would enable companies that REALLY need full-time employees to hire them, so long as they were able to pay the required premium salaries ... or recruit workers who preferred the offered work to paid on-the-side creative-activity....

I suspect this would actually boost the economy, by shifting effort away from BS make-work like processing bogus patents, and toward creative work that would end up having more indirect, ancillary economic value in all sorts of ways.

This may seem a funny thing to think about in the current economic context, when the economy is in such trouble globally. But I consider it obvious that the current economic troubles are "just" a glitch (albeit an extremely annoying one) on the path to a post-scarcity economy. And even so, the government is still giving away money right now to people who are out of work. What if the payment were decreased to people who didn't engage in creative activities, but increased to people who did. Peoples' lives would become richer, as more of them would be more involved in creative activities. And, the human world as a whole would become richer because of all of these new creations.

And this sort of thing will become all the more interesting once robots and AI software eliminate more and more jobs. At that point the market system, unrestricted, would probably create an insane overgrowth of bullshit full-time jobs ... but a part-time creative-activity incentive as I've described could interfere with this dynamic, and nudge things toward a more agreeable situation where people divide their time between genuinely economically useful work, and non-directly-economically-useful but personally-and-collectively-rewarding creative activity.

Furthermore, this would create a culture of creative activity, that would serve us well once robots and AIs eliminate jobs altogether. It would be great if we could greet the Singularity with a culture of collective creativity rather than one characterized by a combination of long hours on useless bullshit jobs, combined with leisure time spent staring at the tube (whether the TV or the YouTube) or playing repetitive shoot-em-up video games ... etc. etc.

OK -- now ... time for me to get back to work ;-)

The Inevitable Increase of Irrationality (more musings on the recent financial crisis)

In a recent email dialogue with my dad, I found myself spouting off again about the possible causes of the recent financial crisis ... and found myself coming to a conclusion that may have some more general implications as we move forward toward Singularity.

Basically, I came to the conclusion that the financial crisis can be interpreted, in part, as a failure of the "rational actor" model in economics.

And furthermore, it's a failure of the rational actor model for an interesting reason: technology is advancing sufficiently that the world is too damn complex for most people to be rational about it, even if they want to be.

And this situation, I suggest, is likely to get worse and worse as technology advances.

Greenspan said something related in an interview shortly after the credit crunch hit: he said was shocked that major banks would deviate so far from rational self-interest.

To the extent this is the case, the recent crisis could be viewed as a failure of the rational-actor model -- and a validation of the need to view socioeconomic systems as complex self-organizing systems, involving psychological, sociological and economic dynamics all tangled together ... a need that will only increase as technology advances and makes it harder and harder for even smart rational-minded people to approximate rational economic judgments.

As a semi-aside: In terms of traditional economists, I'd say Galbraith's perspective probably accounts for this crisis best. He was always skeptical of over-mathematical approaches, and stressed the need to look at economic mechanisms in their social context. And of course, Obama's proposed solution to the problem is strongly Galbraithian in nature (except that Galbraith, not being an elected official, had the guts to call it "socialism" ;-)

Now, there is (of course) a counterargument to the claim that the recent financial crisis indicates a failure of the rational actor model. But one can also make a counter-counterargument, which I find more compelling.

The counterargument is: the banks as institutions were perhaps being irrational, but the individual decision-makers within the banks were being rational in that their incentive structures were asymmetric ... they got big bonuses for winning big, and small penalties for losing big. As a human being who is a banker, taking a huge gamble may be rational, if you get a huge bonus upon winning ... and upon losing, just need to go find another job (a relatively small penalty). So in that sense the individuals working in the banks may have been acting rationally ...

Yet the corporations were not acting rationally: which means that the bank shareholders were the ones not acting rationally, by not constraining the bank managers to put more appropriate incentive structures in place for their employees...

But why were the shareholders acting irrationally?

Well, I suggest that the reason the bank shareholders did not act rationally was, largely, out of ignorance.

Because the shareholders were just too ignorant of the actual risks involved in these complex financial instruments, not to mention of the incentive structures in place within the banks, etc.

We have plenty of legal transparency requirements in place, so that shareholders can see what's going on inside the corporations they invest in. But this is of limited value if the shareholders can't understand what's going on.

So, getting back to rational actor theory: the novel problem that we have here (added on top of the usual human problems of dishonesty, corruption and so forth) may be that, in a world that is increasingly complex (with financial instruments defined by advanced mathematics, for example), being a rational economic actor is too difficult for almost anybody.

The rational-actor assumption fails for a lot of reasons, as many economists have documented in the last few decades ... but this analysis of the current financial crisis suggests that as technology advances, it is going to fail worse and worse.

You could argue that this effect would be counterbalanced by the emergence of an increasingly effective professional "explainer" class who will boil down complex phenomena so that individuals (like bank shareholders) can make judgments effectively.

However, there are multiple problems with this.

For one thing, with modern media, there is so much noise out there that even if the correct explanations are broadcast to the world on the Web, the average person has essentially no way to select them from among the incorrect explanations. OK, they can assume the explanations given in the New York Times are correct ... but of course there is not really any objective and independent press out there, and "believing what the authorities tell you" is a strategy with many known risks.

And, secondly, it's not clear that the journalists of the world can really keep up with developments well enough to give good, solid, objective advice and explanations, even if that is their goal!

So we can expect that as Singularity approaches, the rational-actor model will deviate worse and worse from reality, making economics increasingly difficult to predict according to traditional methods.

Some folks thought that increasing technology would somehow decrease "market friction" and make markets more "efficient" ... in the technical sense of "efficiency," meaning that everyone is paying the mathematically optimal price for the things they buy.

But in fact, increasing technology seems to be increasing "market incomprehensibility" and hence, in at least some important cases, making markets LESS efficient ...

But of course, making markets less "efficient" in the technical sense doesn't necessarily make the economy less efficient in a more general sense.

The economy is, in some senses, becoming fantastically more and more efficient (at producing more and more interesting configurations of matter and mind given less and less usage of human and material resources) ... but it's doing so via complex, self-organizing dynamics ... not via libertarian-style, rational-actor-based free-market dynamics.

Interesting times ahead....

Glocal memory, neural nets, AI, psi

The dichotomy between localized and globalized (distributed) memory in the brain and world is pretty much bullcrap.

For a long time I've had the idea of harmonizing the two, and the Novamente and OpenCog AGI designs incorporate this harmony, as do the speculative brain theories I proposed in my books in the 1990s.

But I never really articulated the concept of global/local ... i.e. glocal ... memory in a general way before, which I've now done in a semi-technical essay called "Glocal Memory"

http://www.goertzel.org/dynapsyc/2008/glocal_memory.htm

I decided to write up the glocal memory stuff now because I'm writing a technical paper on glocal Hopfield neural nets and glocal attention allocation in OpenCog, and I wanted to have something to reference in the paper on glocal-memory-in-general, and there wasn't anything....

In case that's not odd enough for you, I also posted another essay using the glocal memory concept to give a possible model for how psi might work, building on others' ideas regarding filter and transmission models of mind:

http://www.goertzel.org/dynapsyc/2008/glocal_psi.htm

Whoop-di-doodly-dandy-oh

A meta-theory of consciousnes

Let

C = consciousness

P = physical reality

Then, the various theories of consciousness may be placed into 11 categories:

1. C and P are the same
   
2. C is a part of P
3. P is a part of C
4. C and P are parts of something else
5. C and P share a common part but are nonidentical
   
6. C and P are parts of each other (hyperset style)
   
7. C and P are separate -- somehow correlated but not via parthood
   
8. C does not exist, it's an illusion
9. P does not exist, it's an illusion
10. C and P are parts of each other, and also parts of something else
11. C and P are parts of each other, and also parts of something else, which is also a part of them
    

The word "part" should not be overinterpreted in the above, it's just used in a generic and informal sense.

I haven't yet broken this down mathematically, but on quick observation these seem the only logically possible relationships involving C, P and parthood (and allowing for circular parthood).

Each of these theory-categories has some adherents, and if I had more time and tolerance-for-boredom, I could divide existing theories of consciousness into these categories.

The question I want to ask here is: What could the relationship between C and P be, so as to explain why different individuals would propose each of the above 11 theory-categories?

My observation is that there is ONE category of theories that would explain all the 11 theory-categories as different views of a common reality, involving in some cases errors of omission but in no case errors of commission.

This is category 11: that C and P contain each other, and are also contained in something else that is also a part of them

If we posit an underlying model of the general form

C ={C0, P, D}

P = {P0, C, D}

D = {D0, C, P}

then we can see how the other 10 theory-categories would emerge via sins of omission:

1. C and P are the same results from ignoring C0 and P0
   
2. C is a part of P results from ignoring C0
   
3. P is a part of C results from ignoring P0
   
4. C and P are parts of something else ignores that D is a part of C and P, and that C and P are parts of each other
   
5. C and P share a common part but are nonidentical, is correct so long as one allows this common part to be a hyperset (which is C nested within P nested within C ... etc.)
   
6. C and P are parts of each other (hyperset style) results from ignoring D
   
7. C and P are separate -- somehow correlated but not via parthood -- results from projecting C into {C0}, P into {P0}
   
8. C does not exist, it's an illusion results from ignoring C
   
9. P does not exist, it's an illusion results from ignoring P
   
10. C and P are parts of each other, and also parts of something else, results from ignoring that C and P contain D
    

It is thus tempting to conjecture that 11 is the underlying reality, and people who posit the other 10 theories are simply ignoring some of the aspects of reality -- i.e. to use the old metaphor, they are like blind people each feeling a different part of the elephant and concluding that's all there is.

Question for reflection: what is D? ;-)

Ethics as an Attractor

While visiting my father a couple weeks ago, he talked a bit about what he sees as one of the core ideas of religion: the notion of some kind of "universal morality" or "universal moral force-or-field-or-whatever."

While he's not a big fan of the more superstitious aspects of historical or contemporary religions, he does believe there is a universal sense of right versus wrong. For instance, he feels that killing a random person on the street just because one is in a bad mood, is somehow REALLY wrong, not just "wrong according to some person or group's subjective belief system."

My main initial reaction to this idea was not understanding what it might mean. I'm enough of a postmodernist that absolutes don't make much sense to me.

Even "2+2=4" is only true relative to some specific formal system defining the terms involved. And " '2+2=4' is true relative to formal system F " is also only true relative to some metamathematical system -- and so on ... even in mathematics, you never escape the chain of indirections and get to something absolute.

But in the few days after we had that conversation, I thought a bit about what meaning could be assigned to his notion of universal morality within my postmodernist world-view. This is a topic I addressed in the final chapter of The Path to Posthumanity, but not from exactly this perspective: there I was more concerned with enunciating moral principles sufficiently abstract to guide the development of posthumans, rather than with debating the absoluteness or relativity of such principles.

An interesting perspective with pertinence to this issue is Mark Waser's argument that ethical, cooperative behavior (in some form) may be an "attractor" of social systems. Meaning roughly that:

• social systems without something like this are unlikely to survive ... except by adopting some form of ethical, cooperative behavior pattern as a norm
• as a new social system originates and gradually grows, it is likely to evolve its own form of ethical, cooperative behavior

I think this is an interesting perspective, and in the next paragraphs I'll point out some of its limits, and then connect it back to the conversation with my father.

To make the argument more concrete, I'll begin by defining "ethics" in my own special way -- probably not precisely the same as how Mark intends it; but then, in his paper Mark doesn't give a very precisely drawn definition. First of all I'll define a "code of social behavior" as a set of rules, habits or principles that some agent in a society adopts to guide its behavior toward other agents in that society. I'll then define an ethics as a code of social behavior such that

• it occurs in an agent that has the internal flexibility to plausibly adopt the ethics, or not, without causing some sort of immediate disaster for the agent
• it occurs in an agent that is capable of counterfactual reasoning regarding the social situations it regularly encounters (psychologists have various ways to test this, that have been used to study animals for example)
• it involves the agent carrying out behaviors that it reasons (counterfactually) it would not carry out if it did not adhere to the ethics
• it involves the agent carrying out behaviors that the agent believes will benefit other agents in the society

In short, I define an ethics as a code that agents can optionally adopt, and if they adopt it, they know they're taking actions to benefit others, that they reason they wouldn't take in the absence of the code of ethics.

This reminds of a story one of my great-uncles used to tell about my 5-year-old incarnation. He was watching me play with toys, and then my little sister Rebecca came up and asked for one of the toys I was most enjoying. After a moment of reflection, I gave it to her, and then I commented to him that "Sometimes it feels good to do something for someone else that you don't want to do."

Pertaining to Mark Waser's argument, we can then ask whether ethics in this sense is likely to be an attractor. There are two questions here, of course: will the existence of an ethics, any ethics, be an attractor; and will some specific ethics be an attractor. I'll deal mainly with the first question, because if "the existence of ethics" isn't an attractor, then obviously no specific ethics is going to be.

The main limit I see in Waser's argument (as ported to my definition of ethics) is that the argument doesn't seem to apply in cases where one or more of the members of the social system are vastly more intrinsically capable than the others (in relevant ways).

In societies of humans, it could be argued that unethical behavior is ultimately unstable, because the oppressed underclass (with asymmetrically little social power, but roughly equal intrinsic capabilities) will eventually revolt. But the possibility of revolt exists because outside of the scope of the social system, all healthy adult humans have roughly the same level of intrinsic capability.

One can imagine a scenario roughly like the one in H.G. Wells "The Time Machine," where a subset of society that actually is strongly more capable in relevant senses (smarter, saner, stronger) takes control and oppresses the less capable majority. Perhaps any adequately capable individual among the underclass is either killed or taken into the overclass ... and any inadequately capable person among the overclass is either killed or tossed into the underclass. In this kind of scenario, after a certain number of generations, one would have a situation in which there would be pressure for ethics within each class, but not ethics between them.

Nothing like the above has ever happened in human history, of course, and nor is it likely to. However, the case of future AI minds is somewhat different. All humans are built according to the same architecture and have roughly the same amount of intrinsic computational resources, but the same won't necessarily be the case for all AIs.

I see no reason to believe that existence-of-ethics will be attracting in societies involving members with strongly asymmetric capabilities. In fact, it seems it might be easier to frame an alternate argument: that in a society consisting of two groups of radically different degrees of intrinsic capability, the attractor will be

• ethical behavior within the overclass
• ethical behavior within the underclass
• exploitative behavior from the overclass to the underclass

A related situation is human behavior toward "lower animals" -- but this is a different sort of matter because animals don't meet the criteria of ethical agents I laid out above. Adult treatment toward children also doesn't quite fit the mold of this situation, because the intrinsic difference in capability between parents and children reverses as the children grow older (leading to sayings like, "Don't spank your kid too hard; when he grows up he'll be the one choosing your nursing home!").

One thus arrives at the hypothesis that a restricted form of Waser's argument might hold: maybe existence-of-ethics is an attractor in societies composed of agents with roughly equal intrinsic capabilities (in relevant situations).

As to what specific ethical codes may be attractors, it seems to me that is going to depend upon the specifics of the agents and societies. But the general phenomenon of choosing actions for others' benefit, that one knows one would not take in the absence of the ethical code, seems something that could plausibly be argued to serve as an attractor in any society of sufficiently flexible, intelligent organisms with roughly equal intrinsic relevant-capabilities.

Note what the notion of an attractor means here: essentially it means that if you have a society with the right characteristics that *almost* has ethics, then the society will eventually evolve ethics.

Ethics being an attractor doesn't imply that it must be the only attractor; there could be other attractors with very different properties. Arguing that any society with the right characteristics will necessarily evolve into a state supporting ethics, would be a stronger argument.

Another twist on this is obtained by thinking about the difference between the conscious and unconscious minds.

Let's say we're talking about a society consisting of agents with reflective, deliberative capability -- but with a lot of mental habits that aren't easily susceptible to deliberation. This is certainly the situation we humans are in: most of the unconscious behavior-patterns that govern us, are extremely difficult for us to pull into our theater of conscious reflection and reason about ... for a variety of reasons, including the limited capacity of our conscious theater, and the way much unconscious knowledge is represented in the brain, which is only laboriously translatable into the way the conscious theater wants to represent knowledge.

Then, it may be that ethics winds up getting largely packed into the unconscious part of the mind, which is hard to deliberatively reason about. This might happen, for instance, if ethics were largely taught by imitation and reinforcement, rather than by abstract instruction. And this does seem to be how early-childhood ethical instruction happens among humans. We correct a child for doing something bad and reward them for doing something good (reinforcement learning), and we indicate to them real-world everyday-life examples of ethical behavior (both via personal example and via fictional stories, movies and the like). Abstract ethical principles only make sense to them via grounding in this experiential store of ethical examples.

So, if ethics evolves in a society due to its attracting nature, and is memetically propagated largely through unconscious instruction, then in effect what is happening is that in many cases

• the reflective, deliberative mind is thinking about the individual organism and its utility
• the unconscious mind is thinking about the superorganism of the overall society, via the experientially inculcated ethical principles

The voice of the conscience is thus revealed as the voice of the existence-of-ethics attractor that superorganisms (I hypothesize, following Waser) inevitably settle into, assuming their member agents possess certain characteristics.

Where does this leave my dad's notion of a universal ethical force? It doesn't validate any such thing, in quite the sense that my dad seemed to mean it.

However, it does validate the notion that an unconscious sense of ethics may be universal in the sense of being an inevitable mathematical property of any society satisfying certain characteristics.

What does this mean for me, as a reasoning being with an intuitive, unconscious sense of ethics ... and also the deliberative capability to think about this ethical sense ... as well as some ability to modify this sense of ethics if I want to?

Among other things, it reminds me that the deliberative, ratiocinative aspect of the mind probably needs to be a little humbler than it sometimes gets to be, inside us hyperintellectual, nerdy types. The "I" of the deliberative mind is a symbol for the whole mind, rather than constituting the whole mind ... and there may be systematic patterns characterizing which of our mind-patterns get stored in easily-deliberatively-accessible form and which do not. So as frustrating as it can be to those of us in love with ratiocination, if one wants to be maximally rational, one must accept that sometimes "the unconscious knows best" ... even if one can't understand its reasons ... because through observation, imitation and reinforcement, it gained experiential knowledge that possesses a scale and (in some cases) a form that is not ratiocination-friendly (but may yet be rationally considered extremely useful for goal-achievement).

Unfortunately, the unconscious also makes a lot of mistakes and possesses a powerful capacity for tricking itself ... which, all in all, makes the business of being a finite-resources mind with an even-more-acutely-finite-resources deliberative/rational component, an inordinately tricky one.

I personally find these issues slightly less confusing if I view them from the perspective of pattern space. Suppose we consider the universe as set of patterns, governed by a variety of pattern-dynamics rules, including the rule that patterns tend to extend themselves. Different patterns have different degrees of power -- that is, they have differing probabilities of success in extending themselves. The arguments I've given above suggest that ethics, as an abstract pattern for organizing behaviors, is a pattern of considerable power, especially among societies of intelligent entities of roughly comparable capability. In this view there is no "universal moral force" -- the universal force is the tendency of patterns to extend themselves, and ethics as a pattern seems to contain a great deal of this force.

On these issues there are two fairly extreme points of view, which may be compactly summarized (I hope not parodized) as:

1. there is an absolutely correct ethics, an absolute right versus wrong, which may be imperfectly known to humans but which we can hope to better and better discern through various mental and social exercises
2. ethics is purely relative in nature: people adopt ethical codes because they were taught to, or maybe because their genes tell them to (i.e. because their species was taught to, by evolution) ... but there is no fundamental meaning to these ethics rather than social, psychological or evolutionary happenstance

Unlike my father, I never had much attraction to the first perspective. The second perspective has bedeviled me from time to time, yet I've always been nagged by a suspicion there's something deeper going on (yes, of course, someone can attribute this internal nagging to my psychology, my upbringing or my evolutionary heritage!). I don't delude myself I've fully gotten to the bottom of the issue here, but I hope that (building on the ideas of others) I've made a teeny bit of progress.

In What Sense Is "Overcoming Bias" a Good Goal?

This blog post consists of some musings that arose to me a few weeks ago on reading the multi-author blog Overcoming Bias. They were not triggered by the contents of the blog -- though those are sometimes quite interesting -- but merely by the name. My business in this post is to probe the extent to which "overcoming bias" is actually the right way to think about the problem of (as the Welcome page for the Overcoming Bias blog informally puts it) "How can we obtain beliefs closer to reality?"

I think it's clear that "overcoming bias" is important, but I also think it's important to explore and understand the limitations of "overcoming bias" as a methodology for obtaining beliefs that are more useful in achieving one's goals.

(Note that in my own thinking, I tend to more often think in terms of "obtaining beliefs that are more useful in achieving one's goals," rather than in terms of "obtaining beliefs that are closer to reality." In many contexts this just amounts to a nitpick, but it also reflects a significant philosophical distinction. I don't make the philosophical assumption of an objective reality, to which beliefs are to be compared. My philosophical view of beliefs could be loosely considered Nietzschean, though I doubt it agrees with Nietzsche's views in all respects.)

According to Wikipedia,

"
Bias is a term used to describe a tendency or preference towards a particular perspective, ideology or result, especially when the tendency interferes with the ability to be impartial, unprejudiced, or objective. The term biased is used to describe an action, judgment, or other outcome influenced by a prejudged perspective.
"

This definition is worth dissecting because it embodies two different aspects of the "bias" concept, which are often confused in ordinary discourse. I'll call these:

Bias_1: "a tendency or preference toward a particular perspective"

Bias_2: "an instance of Bias_1 that interferes with the ability to be impartial, unprejudiced or objective", which I'll replace with "... interferes with the ability to achieve one's goals effectively."

Bias_1 is, obviously, not necessarily a bad thing.

First of all, the universe we live in has particular characteristics (relative to a universe randomly selected from a sensibly-defined space of possible universes), and being biased in a way that reflects these characteristics may be a good thing.

Secondly, the particular environments and goals that an organism lives in, may have particular characteristics, and it may benefit the organism to be biased in a way that reflects these characteristics.

Now, ideally, an organism would be aware of its environment- or goal- specific biases, so that if its environment or goals change, it can change its biases accordingly. On the other hand, it may be that maintaining this awareness detracts from the organism's ability to achieve goals, if it consumes a lot of resources that could otherwise be spent doing other stuff (even if this other stuff is done in a way that's biased toward the particular environment and goals at hand).

When discussed in a political context, "bias" is assumed to be a bad thing, as in the "especially when" clause of the above Wikipedia definition. Gender bias and racial bias are politically incorrect and according to most modern moral systems, immoral. The reason these biases are considered to be bad is rooted in the (correct, in most cases) assumption that they constitute Bias_2 with respect to the goals that most modern moral systems say we should have.

On the other hand, in cognitive science, bias is not always a bad thing. One may argue, as Eric Baum has done persuasively in What Is Thought?, that the human mind's ability to achieve its goals in the world is largely due to the inductive bias that it embodies, which is placed into it via evolutionary pressure on brain structure. In this context, bias is a good thing. The brain is a general-purpose intelligence, but it is biased to be able to more easily solve some kinds of problems (achieve some kinds of goals) than others. Without this biasing, there's no way a system with the limited computational capacity of the human brain would be able to learn and do all the things it does in the short lifespan of a human organism. The inductive bias that Baum speaks about is largely discussed as Bias_1, but also may in some cases function as Bias_2, because biases that are adaptive in some circumstances may be maladaptive in others.

One might argue that, in the case of evolved human inductive bias, it's the evolutionary process itself that has been less biased, and has (in a relatively unbiased way) evolved brains that are biased to the particular conditions on Earth. However, this is not entirely clear. The evolutionary mechanisms existing on Earth have a lot of particularities that seem adapted to the specific chemical conditions on Earth, for example.

One may argue that, even though we humans are born with certain useful biases, it is to our advantage to become reflective and deliberative enough to overcome these biases in those cases where they're not productive. This is certainly true -- to an extent. However, as noted above, it's also true that reflection and deliberation consume a lot of resources. Any organism with limited resources has to choose between spending its resources overcoming its biases (which may ultimately help it to achieve its goals), and spending its resources achieving its goals in a direct ways.

Furthermore, it's an interesting possibility that resource-constrained minds may sometimes have biases that help them achieve their goals, yet that they are not able to effectively reflect and deliberate on. Why might this be? Because the class of habits that an organism can acquire via reinforcement learning, may not fully overlap with the class of habits that the organism can study via explicit reflective, deliberative inference. For any particular mind-architecture, there are likely to be some things that are more easily learnable as "experientially acquired know-how" than as explicit, logically-analyzable knowledge. (And, on the other hand, there are going to be other things that are more easily arrived at via explicit inference than via experiential know-how acquisition.)

If a certain habit of thought is far more amenable to experiential reinforcement based learning than reflective, logical deliberation, does this mean that one cannot assess its quality, with a view toward ridding it of unproductive biases? Not necessarily. But overcoming biases in these habits may be a different sort of science than overcoming biases in habits that are more easily susceptible to reason. For instance, the best way to overcome these sorts of biases may be to place oneself in a large variety of different situations, so as to achieve a wide variety of different reinforcement signaling patterns ... rather than to reflectively and deliberatively analyze one's biases.

Many of these reflections ultimately boil down to issues of the severely bounded computational capability of real organisms. This irritating little issue also arises when analyzing the relevance of probabilistic reasoning (Bayesian and otherwise) to rationality. If you buy Cox's or de Finetti's assumptions and arguments regarding the conceptual and mathematical foundations of probability theory (which I do), then it follows that a mind, given a huge amount of computational resources, should use probability theory (or do something closely equivalent) to figure out which actions it should take at which times in order to achieve its goals. But, these nice theorems don't tell you anything about what a mind given a small or modest amount of computational resources should do. A real mind can't rigorously apply probability theory to all its judgments, it has to make some sort of heuristic assumptions ... and the optimal nature of these heuristic assumptions (and their dependencies on the actual amount of space and time resources available, and the specific types of goals and environments involved, etc.) is something we don't understand very well.

So, the specific strategy of overcoming Bias_2 by adhering more strictly to probability theory, is interesting and often worthwhile, but not proven (nor convincingly argued) to always the best thing to do for real systems in the real world.

In cases where the answer to some problem can be calculated using probability theory based on a relatively small number of available data items ... or a large number of data items that interrelate in a relatively simply characterizable way ... it's pretty obvious that the right thing for an intelligent person to do is to try to overcome some of their evolutionary biases, which may have evolved due to utility in some circumstances, but which clearly act as Bias_2 in many real-world circumstances. The "heuristics and biases" literature in cognitive psychology contains many compelling arguments in this regard. For instance, in many cases, it's obvious that the best way for us to achieve our goals is to learn to replace our evolved mechanisms for estimating degrees of probability, with calculations more closely reflecting the ones probability theory predicts. Professional gamblers figured this out a long time ago, but the lesson of the heuristics and biases literature has been how pervasive our cognitive errors (regarding probability and otherwise) are in ordinary life, as well as in gambling games.

On the other hand, what about problems a mind confronts that involve masses of different data items, whose interrelationships are not clear, and about which much of the mind's knowledge was gained via tacit experience rather than careful inference or scientific analysis?

Many of these problems involve contextuality, which is a difficult (though certainly not impossible) thing to handle pragmatically within formal reasoning approaches, under severe computational resource constraints.

For these problems, there seem to be two viable strategies for improving one's effectiveness at adapting one's beliefs so as to be able to more adeptly achieve one's goals:

1. Figure out a way to transform the problem into the kind that can be handled using explicit rational analysis
2. Since so much of the knowledge involved was gained via experiential reinforcement-learning rather than inference ... seek to avoid Bias_2 via achieving a greater variety of relevant experiences

So what's my overall takeaway message?

• We're small computational systems with big goals, so we have to be very biased, otherwise we wouldn't be able to achieve our goals
• Distinguishing Bias_1 from Bias_2 is important theoretically, and also important *but not always possible* in practice
• The right way to cure instances of Bias_2 depends to some extent on the nature of the mental habits involved in the bias
• In some cases, diversity of experience may be a better way to remove Bias_2, than explicit adherence to formal laws of rationality
• It is unclear in which circumstances (attempted approximate) adherence to probability theory or other formal laws of rationality is actually the right thing for a finite system to do, in order to optimally achieve its goals
• Heuristically, it seems that adherence to formal laws of rationality generally makes most sense in cases where contextuality is not so critical, and the relevant judgments depend sensitively mainly on a relatively small number of data items (or a large number of relatively-simply-interrelated data items)

In Search of the "Machines of Loving Grace": AI, Robotics and Empathy

Empathy: the ability to feel each others’ feelings. It lies at the core of what makes us human. But how important will it be to the artificial minds we one day create? What are AI researchers doing to imbue their creations with artificial empathy ... and should they be doing more? In short, what is the pathway to the “machines of loving grace” that poet Richard Brautigan foresaw?

The mainstream of AI research has traditionally focused on the more explicitly analytical, intellectual, nerdy aspects of human intelligence: planning, problem-solving, categorization, language understanding. Recent attempts to broaden this focus have focused mainly on creating software with perceptual and motor skills: computer vision systems, intelligent automated vehicles, and so forth. Missing almost entirely from the AI field is the more social and emotional aspects of human intelligence. Chatbots attempting the Turing test have confronted these aspects directly – going back to ELIZA, the landmark AI psychotherapist from the early 1970’s – but these bots are extremely simplistic and have little connection to the main body of work in the AI field.

I think this is a major omission, and my own view is that empathy may be one of the final frontiers of AI. My opinion as an AI researcher is that, if we can crack artificial empathy, the rest of the general-AI problem will soon follow, based on the decades of successes that have already been achieved in problem-solving, reasoning, perception, motorics, planning, cognitive architecture and other areas.

(In my own AI work, involving the Novamente Cognition Engine and the OpenCog Prime system, I’ve sought to explicitly ensure the capability for empathy via multiple coordinated design aspects – but in this blog post I’m not going to focus on that, restricting myself to more general issues.)

Why would empathy be so important for AI? After all, it’s just about human feelings, which are among the least intelligent, most primitively animal-like aspect of the human mind? Well, the human emotional system certainly has its quirks and dangers, and the wisdom of propagating these to powerful AI systems is questionable. But the basic concept of an emotion, as a high-level integrated systemic response to a situation, is critical to functioning of any intelligent system. An AI system may not have the same specific emotions as a human being – particular emotions like love, anger and so forth are manifestations of humans’ evolutionary heritage, rather than intrinsic aspects of intelligence. But it seems unlikely that an AI without any kinds of high-level integrated systemic responses (aka emotions) would be able to cope with the realities of responding to a complex dynamic world in real-time.

A closely related point is the social nature of intelligence. Human intelligence isn’t as individual as we modern Westerners often seem to think: a great percentage of our intelligence is collective and intersubjective. Cognitive psychologists have increasingly realized this in recent decades, and have started talking about “distributed cognition.” If the advocates of the “global brain” hypothesis are correct, then eventually artificial minds will synergize with human minds to form a kind of symbiotic emergent cyber-consciousness. But in order for distributed cognition to work, the minds in a society need to be able to recognize, interpret and respond to each others’ emotions. And this is where empathy comes in.

Mutual empathy binds together social networks: as we go through our lives we are perpetually embodying self-referential emotional equations like

X = I feel pain and that you feel X



Y = I feel that you feel both joy and Y

or mutually-referential ones like

A = I feel happy that you enjoy both this music and B

B = I feel surprised that you feel A

These sorts of equations bind us together: as they unfold through time they constitute much of the rhythm by which our collective intelligence experiences and creates.

So empathy is important: but how does it work?

We don’t yet know for sure ... but the best current thinking is that there are two aspects to how the brain does empathy: inference and simulation. (And I do think this is a lesson for AI: in my own AI designs I deal with these aspects separately, and then address their interaction ... and I do think this is the right approach.)

Inference-wise, empathy has to do with understanding and modeling (sometimes consciously, sometimes unconsciously) what another person must be feeling, based on the cues we perceive and our background knowledge. Psychologists have mapped out transformational rules that help us do this modeling.

Simulative empathy is different: we feel what each other are feeling. A rough analogue is virtualization in computers: running Windows on a virtual machine within Linux; emulating an Activision console within Windows. Similarly, we use the same brain-systems that are used to run ourselves, to run a simulation of another person feeling what they seem to be feeling. And we do this unconsciously, at the body level: even though we don’t consciously notice that sad people have smaller pupils, our pupils automatically shrink when we see a sad person -- a physiological response that synergizes with our cognitive and emotional response to their sadness (and is the technique the lead character uses in Blade Runner to track down androids who lack human feeling). A long list of examples has been explored in the lab already, and we’ve barely scratched the surface yet: people feel disgust when they see others smell bad odor, they feel pain when they see others being pierced by a needle or get electrical shock, they sense touching when they see others being brushed, etc.

Biologists have just started to unravel the neural basis of simulative empathy, which seems to involve brain cells called mirror neurons ... which some have argued play a key role in other aspects of intelligence as well, including language learning and the emergence of the self (I wrote a speculative paper on this a couple years back).

(A mirror neuron is a neuron which fires both when an animal acts and when the animal observes the same action performed by another animal (especially one of the same species). Thus, the neuron "mirrors" the behavior of another animal, as though the observer were itself acting. These neurons have been directly observed in primates, and are believed to exist in humans and in some birds. In humans, brain activity consistent with mirror neurons has been found in the premotor cortex and the inferior parietal cortex.)

So: synergize inference and simulation, and you get the wonderful phenomenon of empathy that makes our lives so painful and joyful and rich, and to a large extent serves as the glue holding together the social superorganism.

The human capacity for empathy is, obviously, limited. This limitation is surely partly due to our limited capabilities of both inference and simulation; but, intriguingly, it might also be the case that evolution has adaptively limited the degree of our empathic-ness. Perhaps an excessive degree of empathy would have militated against our survival, in our ancestral environments?

The counterfactual world in which human empathy is dramatically more intense is difficult to accurately fathom. Perhaps, if our minds were too tightly coupled emotionally, progress would reach the stage of some ant-colony-like utopia and then halt, as further change would be too risky in terms of hurting someone else’s feelings. On the other hand, perhaps a richer and more universal empathy would cause a dramatic shift in our internal architectures, dissolving or morphing the illusion of “self” that now dominates our inner worlds, and leading to a richer way of individually/collectively existing.

One aspect of empathy that isn’t sufficiently appreciated is the way it reaches beyond the touchy-feely sides of human life: for instance it pervades the worlds of science and business as well, which is why there are still so many meetings in the world, email, Skype and WebEx notwithstanding. The main reason professionals fly across the world to hob-nob with their colleagues – in spite of the often exhausting and tedious nature of business travel (which I’ve come to know all too well myself in recent years) -- is because, right now, only face-to-face communication systematically gives enough of the right kind of information to trigger empathic response. In a face-to-face meeting, humans can link together into an empathically-joined collective mind-system, in a way that doesn’t yet happen nearly as reliably via electronically-mediated communications.

Careful study has been given to the difficulty we have empathizing with certain robots or animated characters. According to Mori’s theory of the “uncanny valley” – which has been backed up by brain imaging studies -- if a character looks very close to human, but not close enough, then people will find it disturbing rather than appealing. We can empathize more with the distorted faces of Disney cartoons or manga, than with semi-photo-realistic renditions of humans that look almost-right-but-eerily-off.

To grasp the uncanny valley viscerally, watch one of the online videos of researcher Hiroshi Ishiguro and the “geminoid” robot that is his near-physical-clone -- an extremely lifelike imitation of his own body and its contours, textures and movements. No AI is involved here: the geminoid is controlled by motion-capture apparatus that watches what Ishiguro does and transfers his movements to the robot. The imitation is amazing – until the bot starts moving. It looks close enough to human that its lack of subtle human expressiveness is disturbing. We look at it and we try to empathize, but we find we’re empathizing with a feelingless robot, and the experience is unsettling and feels “wrong.” Jamais Cascio has proposed that existly this kind of reaction may occur to transhumans with body modifications – so from that point of view among others, this phenomenon may be worth attending.

It’s interesting to contrast the case of the geminoid, though, with the experience of interacting with ELIZA, the AI psychotherapist created by Joseph Weizenbaum in 1966. In spite of having essentially no intrinsic intelligence, ELIZA managed to carry out conversations that did involve genuine empathic sharing on the part of its conversaton-partners. (I admit ELIZA didn’t do much for me even back then, but, I encountered it knowing exactly what it was and intrigued by it from a programming perspective, which surely colored the nature of my experience.)

Relatedly, some people today feel more empathy with their online friends than their real-life friends. And yet, I can’t help feel there’s something key lacking in such relationships.

One of the benefits of online social life is that one is freed from the many socio-psychological restrictions that come along with real-world interaction. Issues of body image and social status recede into the background – or become the subject of wild, free-ranging play, as in virtual worlds such as Second Life. Many people are far less shy online than in person – a phenomenon that’s particularly notable in cultures like Japan and Korea where social regulations on face-to-face communcation are stricter.

And the benefits can go far beyond overcoming shyness: for example, a fifty-year-old overweight trucker from Arkansas may be able to relate to others more genuinely in the guise of a slender, big-busted Asian girl with blue hair, a microskirt and a spiky tail... and in Second Life he can do just that.

On the other hand, there’s a certain falsity and emotional distance that comes along with all this. The reason the trucker can impersonate the Asian ingenue so effectively is precisely that the avenues for precise emotional expression are so impoverished in today’s virtual environments. So, the other fifty-year-old trucker from Arkansas whose purple furry avatar is engaged in obscene virtual acts with the Asian babe, has to fill in the gaps left by the simplistic technology – to a large extent, the babe he’s interacting with is a construct of his own mind, which improvises on the cues provided by the signals given by the first trucker.

Of course, all social interaction is constructive in this way: the woman I see when I talk to my wife is largely a construct of my own mind, and may be a different woman than I would see if I were in a different mood (even if her appearance and actions were precisely the same). But text-chat or virtual-world interactions are even more intensely constructive, which is both a plus and a minus. We gain the ability for more complete wish-fulfillment (except for wishes that are intrinsically tied to the physical ... though some people do impressively well as satisfying virtual satisfactions for physical ones), but we lose much of the potential for growing in new directions via empathically absorbing emotional experiences dramatically different from anything we would construct on our own based on scant, sketchy inputs.

It will be interesting to see how the emotional experience of virtual world use develops as the technology advances ... in time we will have the ability to toggle how much detail our avatars project, just as we can now choose whether to watch cartoons or live action films. In this way, we will be able to adjust the degree of constructive wish-fulfillment versus self-expanding experience-of-other ... and of course to fulfill different sorts of wishes than can be satisfied currently in physical or virtual realities.

As avatars become more realistic, they may encounter the uncanny valley themselves: it may be more rewarding to look at a crude, iconic representation of someone else’s face, than a representation that’s almost-there-but-not-quite ... just as with Ishiguro’s geminoids. But just as with the geminoids, the technology will get there in time.

The gaming industry wants to cross the uncanny valley by making better and better graphics. But will this suffice? Yes, a sufficiently perfected geminoid or game character will evoke as much empathy as a real human. But for a robot or game character controlled by AI software, the limitation will probably lie in subtleties of movement. Just like verbal language, the language of emotional gestures is one where it’s hard to spell out the rules exactly: we humans grok them from a combination of heredity and learning. One way to create AIs that people can empathize with will be to make the AIs themselves empathize, and reflect back to people the sorts of emotions that they perceive. Much as babies imitate adult emotions. Envision a robot or game character that watches a video-feed of your face and tailors its responses to your emotions.

Arguably, creating AIs capable of empathy has importance far beyond the creation of more convincing game characters. One of the great unanswered questions as the Singularity looms is how to increase the odds that once our AIs get massively smarter than we are, they still value our existence and our happiness. Creating AIs that empathize with humans could be part of the answer.

Predictably, AI researchers so far have done more with the inferential than the simulative side of empathic response. Selmer Bringsjord’s team at RPI got a lot of press earlier this year for an AI that controls a bot in Second Life, in a way that demonstrates a limited amount of “theory of mind”: the bot watches other characters with a view toward figuring out what they’re aware of, and uses this to predict their behavior and guide its interactions.

But Bringsjord’s bots don’t try to feel the feelings of the other bots or human-controlled avatars they interact with. The creation of AIs embodying simulative empathy seems to be getting very little attention. Rosalind Picard’s Affective Computing Lab at MIT has done some interesting work bringing emotion into AI decision processses but has stopped short of modeling simulative empathy. But I predict this is a subfield that will emerge within the next decade. In fact, it seems plausible that AI’s will one day be far more empathic than humans are – not only with each other but also with human beings. Ultimately, an AI may be able to internally simulate you better than your best human friend, and hence demonstrate a higher degree of empathy. Which will make our games more fun, our robots less eerie, and potentially help make the post-Singularity world a more human-friendly place.

Zarathustra, Plato, Saving Boxes, Oracle Machines and Pineal Antennae

Reading over the conversation I had (with Abram Demski) in the Comments to a prior blog post

http://multiverseaccordingtoben.blogspot.com/2008/10/are-uncomputable-entities-useless-for.html

I was reminded of a conversation I had once with my son Zarathustra when he was 4 years old.

Zar was defending his claim that he actually was omniscient, and explaining how this was consistent with his apparent ignorance on many matters. His explanation went something like this:

"I actually do know everything, Ben! It's just that with all that stuff in my memory, it can take me a really really long time to get the memories out ... years sometimes...."

Of course, Zar didn't realize Plato had been there before (since they didn't cover Plato in his pre-school...).

He also had the speculation that this infinite memory store, called his "saving box", was contained in his abdomen somewhere, separate from his ordinary, limited-scope memories in his brain. Apparently his intuition for philosophy was better than for biology... or he would have realized it was actually in the pineal gland (again, no Descartes in preschool either ;-p).

This reminded me of the hypothesis that arose in the conversation with Abram, that in effect all humans might have some kind of oracle machine in their brains.

If we all have the same internal neural oracle machine (or if, say, we all have pineal-gland antennas to the the same Cosmic Oracle Machine (operated by the ghost of Larry Ellison?)), then we can communicate about the uncomputable even though our language can never actually encapsulate what it is we're talking about.

Terrence McKenna, of course, had another word for these quasi-neural oracle machines: machine-elves ;-)

This means that the real goal of AGI should be to create a software program that can serve as a proper antenna 8-D

Just a little hi-fi sci-fi weirdness to brighten up your day ... I seem to have caught a bad cold and it must be interfering with my thought processes ... or messing up the reception of my pineal antenna ...

P.S.

perhaps some evidence for Zar's saving-box theory:

http://hubpages.com/hub/Cellular-Memories-in-Organ-Transplant-Recipients

Random Memory of a Creative Mind (Paul Feyerabend)

I had a brief but influential (for me: I'm sure he quickly forgot it) correspondence with the philosopher-of-science Paul Feyerabend when I was 19.

I sent him a philosophical manuscript of mine, printed on a crappy dot matrix printer ... I think it was called "Lies and False Truths." I asked him to read it, and also asked his advice on where I should go to grad school to study philosophy. I was in the middle of my second year of grad school, working toward my PhD in math, but I was having second thoughts about math as a career....

He replied with a densely written postcard, saying he wasn't going to read my book because he was spending most of his time on non-philosophy pursuits ... but that he'd glanced it over and it looked creative and interesting (or something like that: I forget the exact words) ... and, most usefully, telling me that if I wanted to be a real philosopher I should not study philosophy academically nor become a philosophy professor, but should study science and/or arts and then pursue philosophy independently.

His advice struck the right chord and the temporary insanity that had caused me to briefly consider becoming a professional philosopher, vanished into the mysterious fog from which it had emerged ...

(I think there may have been another couple brief letters back and forth too, not sure...)

(I had third thoughts about math grad school about 6 months after that, and briefly moved to Vegas to become a telemarketer and Henry-Miller-meets-Nietzsche style prose-poem ranter ... but that's another story ... and anyways I went back to grad school and completed my PhD fairly expeditiously by age 22...)


P.S.

Even at that absurdly young age (but even more so now), I had a lot of disagreements with Feyerabend's ideas on philosophy of science -- but I loved his contentious, informal-yet-rigorous, individualistic style. He thought for himself, not within any specific school of thought or tradition. That's why I wrote to him -- I viewed him as a sort of kindred maverick (if that word is still usable anymore, given what Maverick McCain has done to it ... heh ;-p)

My own current philosophy of science has very little to do with his, but, I'm sure we would have enjoyed arguing the issues together!

He basically argued that science was a social phenomenon with no fixed method. He gave lots of wonderful examples of how creative scientists had worked outside of any known methods.

While I think that's true, I don't think it's the most interesting observation one can make about science ... it seems to me there are some nice formal models you can posit that are good approximations explaining a lot about the social phenomenon of science, even though they're not complete explanations. The grungy details (in chronological order) are at:

• http://www.goertzel.org/dynapsyc/2004/PhilosophyOfScience_v2.htm (included in The Hidden Pattern)
• http://multiverseaccordingtoben.blogspot.com/2008/10/reflections-on-religulous-and.html.
• http://multiverseaccordingtoben.blogspot.com/2008/10/are-uncomputable-entities-useless-for.html

But, one thing I did take from Feyerabend and his friend/argument-partner Imre Lakatos was the need to focus on science as a social phenomenon. What I've tried to do in my own philosophy of science is to pull together the social-phenomenon perspective with the Bayesian-statistics/algorithmic-information perspective on science.... But, as usual, I digress!

hiccups on the path to superefficient financial markets

A political reporter emailed me the other day asking my opinion on the role AI technology played in the recent financial crisis, and what this might imply for the future of finance.

Here's what I told him. Probably it freaked him out so much he deleted it and wiped it from his memory, but hey...

There's no doubt that advanced software programs using AI and other complex techniques played a major role in the current global financial crisis. However, it's also true that the risks and limitations of these software programs were known by many of the people involved, and in many cases were ignored intentionally rather than out of ignorance.

To be more precise: the known mathematical and AI techniques for estimating the risk of complex financial instruments (like credit default swaps, and various other exotic derivatives) all depend on certain assumptions. At this stage, some human intelligence is required to figure out whether the assumptions of a given mathematical technique really apply in a certain real-world situation. So, if one is confronted with a real-world situation where it's unclear whether the assumptions of a certain mathematical technique really apply, it's a human decision whether to apply the technique or not.

A historical example of this problem was the LTCM debacle in the 90's. In that case, the mathematical techniques used by LTCM assumed that the economies of various emerging markets were largely statistical independent. Based on that assumption, LTCM entered into some highly leveraged investments that were low-risk unless the assumption failed. The assumption failed.

Similarly, more recently, Iceland's financial situation was mathematically assessed to be stable, based on the assumption that (to simplify a little bit) a large number of depositors wouldn't decide to simultaneously withdraw a lot of their money. This assumption had never been violated in past situations that were judged as relevant. Oops.

A related, obvious phenomenon is that sometimes humans assigned with the job of assessing risk are given a choice between:

1. assessing risk according to a technique whose assumptions don't really apply to the real-world situation, or whose applicability is uncertain
2. saying "sorry, I don't have any good technique for assessing the risk of this particular financial instrument"

Naturally, the choice commonly taken is 1 rather than 2.

In another decade or two, I'd predict, we'll have yet more intelligent software, which is able to automatically assess whether the assumptions of a certain mathematical technique are applicable in a certain context. That would avoid the sort of problem we've recently seen.

So the base problem is that the software we have now is good at making predictions and assessments based on contextual assumptions ... but it is bad at assessing the applicability of contextual assumptions. The latter is left to humans, who often make decisions based on emotional bias, personal greed and so forth rather than rationality.

Obviously, the fact that a fund manager shares more in their fund's profit than in its loss, has some impact in their assessments. This will bias fund managers to take risks, because if the gamble comes out well, they get a huge bonus, but if it comes out badly, the worst that happens is that they find another job.

My feeling is that these sorts of problems we've seen recently are hiccups on the path to superefficient financial markets based on advanced AI. But it's hard to say exactly how long it will take for AI to achieve the needed understanding of context, to avoid this sort of "minor glitch."

P.S.

After I posted the above, there was a followup discussion on the AGI mailing list, in which someone asked me about applications of AGI to investment.

My reply was:

1)
Until we have a generally very powerful AGI, application of AI to finance will be in the vein of narrow-AI. Investment is a hard problem, not for toddler-minds.

Narrow-AI applications to finance can be fairly broad in nature though, e.g. I helped build a website called stockmood.com that analyzes financial sentiment in news

2)
Once we have a system with roughly adult-human-level AGI, then of course it will be possible to create specialized versions of this that are oriented toward trading, and these will be far superior to humans or narrow AIs at trading the markets, and whomever owns them will win a lot of everybody's money unless the government stops them.

P.P.S.

Someone on a mailing list pushed back on my mention of "AI and other mathematical techniques."

This seems worth clarifying, because the line between narrow-AI and other-math-techniques is really very fuzzy.

To give an indication of how fuzzy the line is ... consider the (very common) case of multiextremal optimization.

GA's are optimization algorithms that are considered AI ... but, is multi-start hillclimbing AI? Many would say so. Yet, some multiextremal optimization algorithms are considered operations research instead of AI -- say, multistart conjugate gradients...

Similarly, backprop NN's are considered AI .. yet, polynomial or exponential regression algorithms aren't. But they pretty much do the same stuff...

Or, think about assessment of credit risk, to determine who is allowed to get what kind of mortgage. This is done by AI data mining algorithms. OTOH it could also be done by some statistical algorithms that wouldn't normally be called AI (though I think it is usually addressed using methods like frequent itemset mining and decision trees, that are considered AI).

Are Uncomputable Entities Useless for Science?

When I first learned about uncomputable numbers, I was profoundly disturbed. One of the first things you prove about uncomputable numbers, when you encounter them in advanced math classes, is that it is provably never possible to explicitly display any example of an uncomputable number. But nevertheless, you can prove that (in a precise mathematical sense) "almost all" numbers on the real number line are uncomputable. This is proved indirectly, by showing that the real number line as a whole has one order of infinity (aleph-one) and the set of all computers has another order of infinite (aleph-null).

I never liked this, and I burned an embarrassing amount of time back then (I guess this was from ages 16-20) trying to find some logical inconsistency there. Somehow, I thought, it must be possible to prove this notion of "a set of things, none of which can ever actually be precisely characterized by any finite description" as inconsistent, as impossible.

Of course, try as I might, I found no inconsistency with the math -- only inconsistency with my own human intuitions.

And of course, I wasn't the first to tread that path (and I knew it). There's a philosophy of mathematics called "constructivism" which essentially bans any kind of mathematical entity whose existence can only be proved indirectly. Related to this is a philosophy of math called "intuitionism."

A problem with these philosophies of math is that they rule out some of the branches of math I most enjoy: I always favored continuous math -- real analysis, complex analysis, functional analysis -- over discrete math about finite structures. And of course these are incredibly useful branches of math: for instance, they underly most of physics.

These continuity-based branches of math also underly, for example, mathematical finance, even though the world of financial transactions is obviously discrete and computable, so one can't possibly need uncomputable numbers to handle it.

There always seemed to me something deeply mysterious in the way the use of the real line, with its unacceptably mystical uncomputable numbers, made practical mathematics in areas like physics and finance so much easier.

Notice, this implicitly uncomputable math is never necessary in these applications. You could reformulate all the equations of physics or finance in terms of purely discrete, finite math; and in most real applications, these days, the continuous equations are solved using discrete approximations on computers anyway. But, the theoretical math (that's used to figure out which discrete approximations to run on the computer) often comes out more nicely in the continuous version than the discrete version. For instance, the rules of traditional continuous calculus are generally far simpler and more elegant than the rules of discretized calculus.

And, note that the uncomputability is always in the background when you're using continuous mathematics. Since you can't explicitly write down any of these uncomputable numbers anyway, they don't play much role in your practical work with continuous math. But the math you're using, in some sense, implies their "existence."

But what does "existence" mean here?

To quote former President Bill Clinton, "it all depends on what the meaning of the word is, is."

A related issue arises in the philosophy of AI. Most AI theorists believe that human-like intelligence can ultimately be achieved within a digital computer program (most of them are in my view overpessimistic about how long it's going to take us to figure out exactly how to write such a program, but that's another story). But some mavericks, most notably Roger Penrose, have argued otherwise (see his books The Emperor's New Mind and Shadows of the Mind, for example). Penrose has argued specifically that the crux of human intelligence is some sort of mental manipulation of uncomputable entities.

And Penrose has also gone further: he's argued that some future theory of physics is going to reveal that the dynamics of the physical world is also based on the interaction of uncomputable entities. So that mind is an uncomputable consequence of uncomputable physical reality.

This argument always disturbed me, also. There always seemed something fundamentally wrong to me about the notion of "uncomputable physics." Because, science is always, in the end, about finite sets of finite-precision data. So, how could these mysterious uncomputable entities ever really be necessary to explain this finite data?

Obviously, it seemed tome, they could never be necessary. Any finite dataset has a finite explanation. But the question then becomes whether in some cases invoking uncomputable entities is the best way to explain some finite dataset. Can the best way of explaining some set of, say, 10 or 1000 or 1000000 numbers be "This uncomputable process, whose details you can never write down or communicate in ordinary language in a finite amount of time, generated these numbers."

This really doesn't make sense to me. It seems intuitively wrong -- more clearly and obviously so than the notion of the "existence" of uncomputable numbers and other uncomputable entities in some abstract mathematical sense.

So, my goal in this post is to give a careful explanation of why this wrong. The argument I'm going to give here could be fully formalized as mathematics, but, I don't have the time for that right now, so I'll just give it semi-verbally/semi-mathematically, but I'll try to choose my words carefully.

As often happens, the matter turned out to be a little subtler than I initially thought it would be. To argue that uncomputables are useless for science, one needs some specific formal model of what science itself is. And this is of course a contentious issue. However, if one does adopt the formalization of science that I suggest, then the scientific uselessness of uncomputables falls out fairly straightforwardly. (And I note that this was certainly not my motivation for conceiving the formal model of science I'll suggest; I cooked it up a while ago for quite other reasons.)

Maybe someone else could come up with a different formal model of science that gives a useful role to uncomputable entities ... though one could then start a meta-level analysis of the usefulness of this kind of formal model of science! But I'll defer that till next year ;-)

Even though it's not wholly rigorous math, this is a pretty mathematical blog post that will make for slow reading. But if you have suitable background and are willing to slog through it, I think you'll find it an interesting train of thought.

NOTE: the motivation to write up these ideas (which have been bouncing around in my head for ages) emerged during email discussions on the AGI list with a large group, most critically Abram Demski, Eric Baum and Mark Waser.

A Simple Formalization of the Scientific Process

I'll start by giving a simplified formalization of the process of science.

This formalization is related to the philosophy of science I outlined in the essay http://www.goertzel.org/dynapsyc/2004/PhilosophyOfScience_v2.htm (included in The Hidden Pattern) and more recently extended in the blog post http://multiverseaccordingtoben.blogspot.com/2008/10/reflections-on-religulous-and.html. But those prior writing consider many aspects not discussed here.

Let's consider a community of agents that use some language L to communicate. By a language, what I mean here is simply a set of finite symbol-sequences ("expressions"), utilizing a finite set of symbols.

Assume that a dataset (i.e., a finite set of finite-precision observations) can be expressed as a set of pairs of expressions in the language L. So a dataset D can be viewed as a set of pairs


((d11, d12), (d21,d22) ,..., (dn1,dn2))

or else as a pair D=(D1,D2) where

D1=(d11,...,dn1)
D2=(d12,...,dn2)

Then, define an explanation of a dataset D as a set E_D of expressions in L, so that if one agent A1 communicates E_D to another agent A2 that has seen D1 but not D2, nevertheless A2 is able to reproduce D2.

(One can look at precise explanations versus imprecise ones, where an imprecise explanation means that A2 is able to reproduce D2 only approximately, but this doesn't affect the argument significantly, so I'll leave this complication out from here on.)

If D2 is large, then for E_D to be an interesting explanation, it should be more compact than D2.

Note that I am not requiring E_D to generate D2 from D1 on its own. I am requiring that A2 be able to generate D2 based on E_D and D1. Since A2 is an arbitrary member of the community of agents, the validity of an explanation, as I'm defining it here, is relative to the assumed community of agents.

Note also that, although expressions in L are always finitely describable, that doesn't mean that the agents A1, A2, etc. are. According to the framework I've set up here, these agents could be infinite, uncomputable, and so forth. I'm not assuming anything special about the agents, but I am considering them in the special context of finite communications about finite observations.

The above is my formalization of the scientific process, in a general and abstract sense. According to this formalization, science is about communities of agents linguistically transmitting to each other knowledge about how to predict some commonly-perceived data, given some other commonly-perceived data.

The (Dubious) Scientific Value of the Uncomputable

Next, getting closer to the theme of this post, I turn to consider the question of what use it might be for A2 to employ some uncomputable entity U in the process of using E_D to generate D2 from D1. My contention is that, under some reasonable assumptions, there is no value to A2 in using uncomputable entities in this context.

D1 and E_D are sets of L-expressions, and so is D2. So what A2 is faced with, is a problem of mapping one set of L-expressions into another.

Suppose that A2 uses some process P to carry out this mapping. Then, if we represent each set of L-expressions as a bit string (which may be done in a variety of different, straightforward ways), P is then a mapping from bit strings into bit strings. To keep things simple we can assume some maximum size cap on the size of the bit strings involved (corresponding for instance to the maximum size expression-set that can be uttered by any agent during a trillion years).

The question then becomes whether it is somehow useful for A2 to use some uncomputable entity U to compute P, rather than using some sort of set of discrete operations comparable to a computer program.

One way to address this question is to introduce a notion of simplicity. The question then becomes whether it is simpler for A2 to use U to compute P, rather than using some computer program.

And this, then, boils down to one's choice of simplicity measure.

Consider the situation where A2 wants to tell A3 how to use U to compute P. In this case, A2 must represent U somehow in the language L.

In the simplest case, A2 may represent U directly in the language, using a single expression (which may then be included in other expressions). There will then be certain rules governing the use of U in the language, such that A2 can successfully, reliably communicate "use of U to compute P" to A3 only if these rules are followed. Call this rule-set R_U. Let us assume that R_U is a finite set of expressions, and may also be expressed in the language L.

Then, the key question is whether we can have

complexity(U) < complexity(R_U)

That is, can U be less complex than the set of rules prescribing the use of its symbol S_U within the community of agents?

If we say NO, then it follows there is no use for A2 to use U internally to produce D2, in the sense that it would be simpler for A2 to just use R_U internally.

On the other hand, if we say YES, then according to the given complexity measure, it may be easier for A2 to internally make use of U, rather than to use R_U or something else finite.

So, if we choose to define complexity in terms of complexity of expression in the community's language L, then we conclude that uncomputable entities are useless for science. Because, we can always replace any uncomputable entity U with a set of rules for manipulating the symbol S_U corresponding to it.

If you don't like this complexity measure, you're of course free to propose another one, and argue why it's the right one to use to understand science. In a previous blog post I've presented some of the intuitions underlying my assumption of this "communication prior" as a complexity measure underlying scientific reasoning.

The above discussion assumes that U is denoted in L by a single symbolic L-expression S_U, but the same basic argument holds if the expression of U in L is more complex.

What does all this mean about calculus, for example ... and the other lovely uses of uncomputable math to explain science data?

The question comes down to whether, for instance, we have

complexity(real number line R) <>

If NO, then it means the mind is better off using the axioms for R than using R directly. And, I suggest, that is what we actually do when using R in calculus. We don't use R as an "actual entity" in any strong sense, we use R as an abstract set of axioms.

What would YES mean? It would mean that somehow we, as uncomputable beings, used R as an internal source of intuition about continuity ... not thus deriving any conclusions beyond the ones obtainable using the axioms about R, but deriving conclusions in a way that we found subjectively simpler.

A Postcript about AI

And, as an aside, what does all this mean about AI? It doesn't really tell you anything definitive about whether humanlike mind can be achieved computationally. But what it does tell you is that, if

• humanlike mind can be studied using the communicational tools of science (that is, using finite sets of finite-precision observations, and languages defined as finite strings on finite alphabets)
• one accepts the communication prior (length of linguistic expression as a measure of complexity)

then IF mind is fundamentally noncomputational, science is no use for studying it. Because science, as formalized here, can never distinguish between use of U and use of S_U. According to science, there will always be some computational explanation of any set of data, though whether this is the simplest explanation depends on one's choice of complexity measure.

Cosmic, overblown Grand Unified Theory of Development

In the 80's I spent a lot of time in the "Q" section of various libraries, which hosted some AI books, and a lot of funky books on "General Systems Theory" and related forms of interdisciplinary scientifico-philosophical wackiness.

GST is way out of fashion in the US, supplanted by Santa Fe Institute style "complexity theory" (which takes the same basic ideas but fleshes them out differently using modern computer tech), but I still have a soft spot in my heart for it....

Anyway, today when I was cleaning out odd spots of the house looking for a lost item (which I failed to find and really need, goddamnit!!) I found some scraps of paper that I scribbled on a couple years back while on some airline flight or another, sketching out the elements of a general-systems-theory type Grand Unified Theory of Development ... an overall theory of the stages of development that complex systems go through as they travel from infancy to maturity.

I'm not going to type in the whole thing here right now, but I made a table depicting part of it, so as to record the essence of the idea in some nicer, more permanent form than the fading dirty pieces of notebook paper....

The table shows the four key stages any complex system goes through, described in general terms, and then explained in a little more detail in the context of two examples: the human (or humanlike) mind as it develops from infancy to maturity, and the maturity of life from proto-life up into its modern form.

I couldn't get the table to embed nicely in this blog interface, so it's here as a PDF:

Table
of the Universal Stages of Development
as Manifested in Mind and Life

This was in fact the train of thought that led to two papers Stephan Bugaj and I wrote over the last couple years, on the stages of cognitive development of uncertain-inference based AI systems, and the stages of ethical development of such AI systems. While not presented as such in those papers, the stages given there are really specialized manifestations of the more general stages outlined in the above table.

Stephan and I are (slowly) brewing a book on hyperset models of mind and reality, which will include some further-elaborated, rigorously-mathematized version of this general theory of development...

Long live General Systems thinking ;-)

Parable of the Researcher and the Tribesman

I run an email discussion list on Artificial General Intelligence, which is often interesting, but lately the discussions there have been more frustrating than fascinating, unfortunately.

One recent email thread has involved an individual repeatedly claiming that I have not presented any argument as to why my designs for AGI could possibly work.

When I point to my published or online works, which do present such arguments, this individual simply says that if my ideas make any sense, I should be able to summarize my arguments nontechnically in a few paragraphs in an email.

Foolishly, I managed to get sufficiently annoyed at this email thread that I posted a somewhat condescending and silly parable to the email list, which I thought I'd record here, just for the heck of it....

What I said was:

In dialogues like this, I feel somewhat like a medical researcher talking to a member of a primitive tribe, trying to explain why he thinks he has a good lead on a potential drug to cure a disease. Imagine a dialogue like this:

• RESEARCHER: I'm fairly sure that I'll be able to create a drug curing your son's disease within a decade or so
• TRIBESMAN: Why do you believe that? Have you cured anyone with the drug?
• RESEARCHER: No, in fact I haven't even created the drug yet
• TRIBESMAN: Well, do you know exactly how to make the drug?
• RESEARCHER: No, not exactly. In fact there is bound to be some inventive research involved in making the drug.
• TRIBESMAN: Well then how the hell can you be so confident it's possible?
• RESEARCHER: Well I've found a compound that blocks the production of the protein I know to be responsible for causing the disease. This compound has some minor toxic effects in rats, but it's similar in relevant respects to other compounds that have shown toxic effects in rats, and then been minorly modified to yield variant compounds with the same curative impacts without toxic effects
• TRIBESMAN: So you're saying it's cured the same disease in rats?
• RESEARCHER: Yes, although it also makes the rats sick ... but if it didn't make them sick, it would cure them. And I'm pretty sure I know how to change it so as to make it not make the rats sick. And then it will cure them.
• TRIBESMAN: But my son is not a rat. Are you calling my son a rat? You don't seem to understand what a great guy my son is. All the women love him. His winky is twice as long as yours. What does curing a rat have to do with curing my son? And it doesn't even cure the rat. It makes him sick. You just want to make my son sick.
• RESEARCHER: Look, you don't understand. If you look at all the compounds in that class, you'll see there are all sorts of ways to modify them to avoid these toxic effects.
• TRIBESMAN: So you're saying I should believe you because you're a big important scientist. But your drug hasn't actually cured anyone. I don't believe it'll possibly work. People come by here all the time trying to sell me drugs and they never work. Those diet pill were supposed to make my wife 100 pounds thinner, but she still looks like a boat.
• RESEARCHER: I'm not responsible for the quacks who sold you diet pills
• TRIBESMAN: They had white lab coats just like yours
• RESEARCHER: Look, read my research papers. Then let's discuss it.
• TRIBESMAN: I can't read that gobbledygook. Do all the other researchers agree with you?
• RESEARCHER: Some of them do, some of them don't. But almost all of them who have read my papers carefully think I at least have a serious chance of turning my protein blocker into a cure. Even if they don't think it's the best possible approach.
• TRIBESMAN: So all the experts don't even agree, and you expect me to take you seriously?
• RESEARCHER: Whatever. I'll talk to you again when I actually have the cure. Have a nice few years.
• TRIBESMAN: We won't need your cure by then, Mr. Scientist. We're curing him with leeches already.

That just about sums it up....

The point is, the researchers's confidence comes from his intuitive understanding of a body of knowledge that the tribesman cannot appreciate due to lack of education.

The tribesman says "you haven't cured anyone, therefore you know nothing about the drug" ... but the researcher has a theoretical framework that lets him understand something about the drug's activity even before trying it on people.

Similarly, some of us working on AGI have a theoretical framework that lets us understand something about our AGI systems even before they're complete ... this is what guides our work building the systems. But conveying our arguments to folks without this theoretical framework is, unfortunately, close to impossible.... If I were to write some sort of popular treatment of my AGI work, the first 75% of it would have to consist of a generic explanation of background ideas (which is part of the reason I don't take the time to write such a thing ... it seems like an awful lot of work!!).

Obvious stuff, of course. I'm metaphorically kicking myself for burning half an hour in this sort of absurd email argument tonight ... gotta be more rigorous about conserving my time and attention, there's a lot of work to be done!!!

Reflections on "Religulous" ... and introducing the Communication Prior

I saw the documentary Religulous w/ my kids last night (well, the two who still live at home) ... it's a sort of goofball documentary involving comedian Bill Maher interviewing people with absurd religious beliefs (mostly focusing on Christians, Jews and Muslims, with a few other oddities like a Scientologist street preacher and an Amsterdam cannabis-worshipper) ...

This blog post records some of my random reactions to the movie, and then at the end gets a little deeper and presents a new theoretical idea that popped into my head while thinking about the difficulty of making a really sound intellectual refutation of common religious beliefs.

The new theoretical idea is called the Communication Prior ... and the crux is the notion that in a social group, the prior probability of a theory may be defined in terms of the ease with which one group member can rapidly and accurately communicate the theory to another. My suggestion is that the Communication Prior can serve as the basis for a pragmatic everyday interpretation of Occam's Razor (as opposed to the Solomonoff-Levin Prior, which is a formal-computer-science interpretation). This is important IMHO because science ultimately boils down to pragmatic everyday social phenomena not formal mathematical phenomena.

Random Reactions to Religulous

First a bit about Religulous, which spurred the train of thought reported here....

Some of the interviews in the movie were really funny -- for instance a fat Puerto Rican preacher named Jesus who claims to literally be the Second Coming of Christ, and to have abolished sin and hell ...

and as a whole the interviews certainly made Maher's point that all modern religions are based on beliefs that seem bizarre and twisted in the light of the modern scientific world-view ... the talking snake in the Garden of Eden ... Judgment Day when God comes to Earth and sorts the goodies from the baddies ... the notion that rapture will come only when the Muslims have finally killed all the Jews ... etc. etc. etc. etc. etc. ...

Some interesting historical tidbits were presented as well, e.g. the Egyptian figure Horus, who well predated Christ and whose life-story bears remarkable similarities to the Biblical tale of Jesus....

I've never been a huge fan of stand-up comedians; and among comedians Maher doesn't really match my taste that well ... he's not outrageous or absurd enough ... so I got a bit weary of his commentary throughout the film, but I felt the interviews and interspersed film and news snippets were well-done and made his point really well.

Of course, it's a damn easy point to make, which was part of his point: Of course all religions ancient and modern have been based on bizarre, wacky, impossible-for-any-sane-person-to-believe, fictional-sounding ideas...

One point that came up over and over again in his dialogues with religious folks was his difference with them over the basic importance (or lack thereof) of faith. "Why," he kept asking, "is faith a GOOD thing? Why is it a good thing to believe stuff that has no evidence in favor of it? Why is it a good thing to believe stuff that makes no sense and contradicts observation and apparent reality?"

The answer the religious folks invariably give him is something like "Faith is a good thing because it saved my life."

Dialogue like: "I used to be a Satan worshipper and wasted decades of my life on sex and drugs ... Getting saved by Jesus saved my life blahblaa..."


Religion and Politics: Egads!


Maher's interview with a religious fundamentalist US Senator is a bit disturbing. Indeed, to have folks who believe Judgment Day is nigh, in charge of running the most powerful country in the world, is, uh, scary....

And note that our outgoing President, W Bush, repeatedly invokes his religious beliefs in justifying his policies. He explicitly states that his faith in God is the cornerstone of his policies. Scary, scary, scary. I don't want to live in a society that is regulated based on someone's faith in a supernatural being ... based on someone's faith in the literal or metaphorical truth of some book a bunch of whacked-out, hallucinating Middle-Easterners wrote 2000 years ago....

As Maher points out, this is a completely senseless and insane basis for a modern society to base itself on....


Maher's Core Argument

I don't expect Maher's movie to un-convert a substantial number of religious folks...

Their natural reaction will be: "OK, but you just interviewed a bunch of kooks and then strung their kookiest quotes together."

Which is pretty much what he did ... and in a way that may well be compelling as a tool for helping atheists feel more comfortable publicly voicing their beliefs (which I imagine was much of his purpose) ...

And it has to be noted that a deep, serious, thorough treatment of the topic of religion and irrationality would probably never get into movie theaters.

Modern culture, especially US culture but increasingly world culture as well, has little time for deep rational argumentation. Al Gore made this book quite nicely in his book The Assault on Reason ... which however not that many people read (the book contained too much rational argumentation...).

So it's hard to fault Maher's film for staying close to the surface and presenting a shallow argument against religion ... this is the kind of argument that our culture is presently willing to accept most easily ... and if atheists restricted themselves to careful, thorough, reflective rational arguments, the result would be that even fewer people would listen to them than is now the case....

Maher's argument is basically: All religions have absurd, apparently-delusional, anti-scientific beliefs at their core ... and these absurd beliefs are directly tied to a lot of bad things in the world ... Holy Wars and so forth ....

He also, correctly, traces the bizarre beliefs at the heart of religions to altered brain-states on the part of religious prophets.

As he notes, if someone today rambled around telling everyone they'd been talking to a burning bush up on a hill, they'd likely get locked into a mental institution and force-fed antipsychotics. Yet, when this sort of experience is presented as part of the history of religion, no one seems to worry too much -- it's no longer an insane delusion, it's a proper foundation for the government of the world ;-p

What Percentage of the Population Has a World View Capable of Sensibly Confronting the Singularity?

One thing that struck me repeatedly when listening to Maher's interviews was:

Wow, given all the really HARD issues the human races during this period of rapidly-approaching Singularity ... it's pathetic that we're still absorbed with these ridiculous debates about talking snakes and Judgment Day and praying to supreme beings ... egads!!!

While a digression from this blog post, this is something I think about a lot, in the context of trying to figure out the most ethical and success-probable approach to creating superhuman AI....

On the one hand, due to various aspects of human psychology, I don't trust elitism much: the idea of a small group of folks (however gifted and thoughtful) creating a superhuman AI and then transforming the world, without broader feedback and dialogue, is a bit scary....

On the other hand, I've got to suspect that folks who believe in supreme beings, Judgment Day, jihad, reincarnation and so forth are not really likely to have much useful contribution to the actual hard issues confronting us as Singularity approaches....

Of course, one can envision a lot of ways of avoiding the difficulties alluded to in the prior two paragraphs ... but also a lot of ways of not avoiding them....

One hope is that Maher's movie and further media discourse legitimizing atheism will at least somewhat improve the intellectual level of broad public conversation ... so that, maybe, in a decade or so it won't be political suicide for a US Senatorial candidate to admit they're not religious or superstitious, for example...

On the other hand, it may well eventuate that this process of de-superstitionizing the world will be damn slow compared to the advent of technology ...

But, that's a topic for another lengthy blog post, some other weekend....


The Issues Posed by the "Problem of Induction" and the Philosophy of Science for the Argument Against Religion

Now I'll start creeping, ever so slowly, toward the more original intellectual content of this post, by asking: What might a more deeply reasoned, reflective argument against religion look like?

This topic is actually fairly subtle, because it gets at deep issues in the philosophy of science ... such as I reviewed in an essay a few years ago (included in my 2006 book The Hidden Pattern)...

Although Maher talks a lot about scientific evidence ... and correctly points out that there is no scientific evidence for the various kooky-sounding claims at the core of modern religions ... he doesn't seem to have thought much about the nature of scientific evidence itself. (Which is no surprise as he's a professional comedian and actor ... but of course, he's now a self-styled commentator on politics, science and religion, so....)

Evidence, in the sense of raw data, is not disputed that often among scientists -- and even religious folks don't dispute raw data collected by scientists that often. Statements like "this laboratory instrument, at this point in time, recorded this number on its dial" are not oft disputed. Sometimes argumentation may be made that not enough data were recorded to evaluate an empirical statement like the above (say, the temperature in the room, or the mind-state of the lab assistant, were not recorded): but this still isn't really an argument that the data are wrong, more an argument that the data are too incomplete to draw useful conclusions from them.

(The only area of research I know where raw data is routinely disputed is psi ... which I already addressed in a prior blog post.)

But the step from raw items of evidence to theory is a big one -- a bigger one than Maher or most naively-pro-science advocates care to admit.

This of course relates to the uncomfortable fact that the Humean problem of induction was never solved.

As Maher points out repeatedly in his film, we just don't really know anything for sure ... and it appears that by the basic logic of the universe and the nature of knowledge itself, we never can.

What he doesn't point out (because it's not that kind of movie) is that without making some kind of background assumptions (going beyond the raw evidence collected), we also can't really make probability estimates, or probabilistic predictions about the outcomes of experiments or situations.

Given a set of observations, can we predict the next observations we'll see? Even probabilistically? As Hume pointed out, we can do so only by making some background assumptions.

For instance, we can adopt the Occam's Razor heuristic and assume that there will be some simple pattern binding the past observations to the future ones.... But that begs the question: what is the measure of simplicity?

Hume says, in essence, that the relevant measure of simplicity is human nature.

But this conclusion may, initially, seem a bit disturbing in the context of the religion vs. science dichotomy.

Because, human nature in in many ways, not to put it too tactlessly, more than a bit fucked-up.

Maher doesn't review the evidence in this regard, but he does allude to it, e.g interviewing the discoverer of the "God gene" ... the point is: it seems to be the case that religious experience and religious delusions are deeply tied to intrinsic properties of the human brain.

What this suggests is that the reason religion is so appealing to people is precisely that it is assigned a high prior probability by their Humean "human nature" ... that our brain structure, which evolved in superstitious pre-civilized societies, biases us towards selecting theories that not only explain our everyday empirical observations, but also involve talking animals, voices speaking from the sky, tribalism, physical rewards or punishments for moral transgressions, and so forth...

So when Maher says that "it's time for us to grow up" and let go of these ancient religious superstitions and just be rational and scientific ... two big problems initially appear to arise, based on cursory consideration of the philosophy of science:

• There is no such thing as "just being rational" ... applying rationality to real observations always involves making some background assumptions
• The ancient religious superstitions are closely related to patterns wired into our brains by evolution ... which are naturally taken by us as background assumptions...
  

So when he asks folks to drop their religious beliefs, is Maher really asking folks to self-modify their brains so as not to apply prior distributions supplied by evolution (which has adapted our cognitive patterns to superstitious, tribal society), and to instead apply prior distributions supplied by the scientific and rationalist tradition...?

If so, that would seem a really tough battle to fight. If this were the case, then essentially, the transcendence of religious superstitions would require a kind of cognitive transhumanism.

Fortunately, though I don't think the situation is quite that bad. Cognitive transhumanism (which I define as the attempt to go beyond innately-human patterns of thinking) certainly can be a huge help in the transcendence of superstitions, but it's not strictly necessary.

It appears to me that it's enough "just" to get people to think more clearly about the relationship between their theories and ideas, their community, and their community's collective observations. If people understand this relationship clearly, then it's not actually necessary for them to transcend their various superstition-oriented human biases in order for them to go beyond naive religious ideas.

To elaborate on this point further I'll need to get technical for a moment and introduce a bit of Bayesian statistics and algorithmic information theory...

The Communication Prior

I'll now shift from philosophical babbling to basic math for a few paragraphs.

Recall the basics of Bayes Theorem... . Setting T for "theory" and E for "evidence", it says:

P(T|E) = P(T) P(E|T)/P(E)
... i.e., it says that a person's subjective probability that a theory T is true given that they receive evidence E, should be equal to their prior probability that T is true times the probability that they would receive evidence E if hypothesis T were true, divided by the probability of E (and the latter is usually found by summing over the weighted conditional probabilities given all potential theories).

It is critical to note that, according to Bayes rule, one's conclusion about the probability of theory T given evidence E depends upon one's prior assignment of probabilities.

Now, a real mind with computational limitations cannot always apply Bayes rule accurately ... so the best we can do is approximate.

(Some cognitive theorists, such as Pei Wang, argue that a real mind shouldn't even try to approximate Bayes rule, but should utilize a different logic specially appropriate for cognitive systems with severe resource limitations ... but I don't agree with this and for the purpose of this blog post will assume it's not the case.)

But even if a mind has enough computational resources to apply Bayes rule correctly, there remains the problem of how to arrive at the prior assignment of probabilities?

The most commonsensical way is to use Occam's Razor, the maxim stating that simpler hypotheses should be considered a priori more probable. But this also leads to some subtleties....

The Occam maxim has been given mathematical form in the Solomonoff-Levin universal prior, which says very roughly that the probability of a hypothesis is higher if the computer-programs for computing that hypothesis are shorter (yes, there's more to it, so look it up if you're curious).

Slightly more rigorously, Wikipedia notes that:

The universal prior probability of any prefix p of a computable sequence x is the sum of the probabilities of all programs (for a universal computer) that compute something starting with p. Given some p and any computable but unknown probability distribution from which x is sampled, the universal prior and Bayes' theorem can be used to predict the yet unseen parts of x in optimal fashion.

Note in the above quote that the probability of a program may be estimated as the probability that the program is found by randomly selecting bits in the program-defining section of the memory of a computer.

Anyway: That's very nice for mathematicians, but it doesn't help us much in everyday life ... because even if we wanted to apply this kind of formalization in everyday life (say, to decide an issue like evolution vs. creationism), the mapping of real-world situations into mathematical formalisms is itself highly theory-laden....

So what we really need is not just a mathematical formalization of a universal prior, but a commonsensical formalization of a prior that is helpful for everyday human situations (even if not truly universal).

One suggestion I have is to use Solomonoff's core idea here, but interpret it a bit differently, in terms of everyday human communicational operations rather than mathematical, abstracted machine operations.

Paraphrasing the above quoted text, I propose that

The communicational prior probability of any prefix p of a computable sequence x, relative to a social group G and a body of evidence E, is the sum of the communicational probabilities (calculated relative to G and E) of all programs that compute something starting with p.

But how then to compute the communicational probability of a program relative to a social group G and body of evidence E?

As the name indicates, this is defined, not in terms of bit-flipping, but in terms of communication within the group.

I define the communicational probability of a program p, as being proportional to the average amount of time it would take a randomly chosen member A of group G to communicate p to another randomly chosen member B of group G, with sufficient accuracy that G can then evaluate the outputs of p on randomly selected inputs drawn from E.

(The assumption is that A already knows how to evaluate the program on inputs drawn from E.)

One can also bake a certain error rate into this definition, so that G has to be able to correctly evaluate the outputs of p only on a certain percentage of inputs drawn from E.

This defines what I suggest to call the Communication Prior.

A variant would be the communication-and-testing probability of a program p, definable as being proportional to the average, for randomly chosen members A and B in the social group such that A already knows how to evaluate p on inputs in E, of

• the amount of time it would take A to communicate p to B, with sufficient accuracy that B can then evaluate the outputs of p on randomly selected inputs drawn from E
• the amount of time it actually takes B to evaluate p on a randomly selected element of E

(One can of course weight the two terms in this average, if one wants to.)

Taking a bit of terminological liberty, I will also group this communication-testing variant as being under the umbrella of the "Communication Prior."

Pragmatically, what does this mean about theories?

Roughly speaking, it means that the a priori probability of a theory (i.e. the "bias toward" a theory) has to do with ease of effectively communicating that theory within a social group ... and (in the communication-testing variant), the ease of effectively communicating how to efficiently apply the theory.

Of course, the a priori probability theory doesn't tell you how good a theory is. Communicating a theory may be very simple, but so what ... unless the theory explains something. But the "explanation" part is taken care of in Bayes Rule, in the P(E | T) / P(E) fraction. If the observed evidence is not surprisingly likely given the assumption of the theory, then this fraction will be small.

The Communication Prior is similar in spirit to the Solomonoff-Levin Universal Prior ... but it's not about formal, mathematical, theoretical systems, it's about real-world social systems, such as human communities of scientists. In terms of philosophy of science, this is sort-of a big deal, as it bridges the gap between formalist and social-psychology-based theories of science.

What's the Take-Away from All That Techno-babble?

So, roughly speaking, the nontechnical take-away from the above technical excursion should be the following suggestion:

A theory should be considered good within a social group, to the extent that it explains the evidence better than it would explain a bunch of randomly selected evidence -- and it's reasonably rapid to effectively communicate, to others in the group, information about how to efficiently apply the theory to explain the available evidence.

This may seem simple or almost obvious, but it doesn't seem to have been said before, in quite so crisp of a way.

(In my prior essay on philosophy of science, I left off without articulating any sort of specific simplicity measure: the Communication Prior fills in that gap, thus bringing the ideas in that essay closer to practical applicability.)

Consider for instance the evolution vs. creationism argument. For my new suggestion to favor evolution over creationism, what would have to be true?

Whether the simple essential core of creationism or evolution is easier to communicate within a human social group, really depends on the particular social group.

However, the simple essential core of creationism does an extremely bad job of explaining why the observed body of evidence (e.g. the fossil record) is more likely than a lot of other possible bodies of evidence.

To make a version of creationism that would explain why the observed body of evidence is particularly likely, one would need to add a heck of a lot of special-pleading-type explanations onto the essential core of creationism. This is because creationism does not effectively compress or compactify the body of observed data.

So, to get a version of creationism that is equally explanatory of the particulars of the evidence as evolution, one needs to make a version of creationism that takes a long time to communicate.

Conclusion: creationism is worse than evolution.

(OK, we don't really need to go through so much complexity to get to such an obvious conclusion! But I'm just using that example to make a more general point, obviously.)

Why Is Religion a Bad Idea?

Getting back to the initial theme of this overlong, overdiverse blog post, then: why is religion a bad idea?

Because we should judge our theories using Bayes rule with a communication prior ... or in other words, by asking that they explain the particulars of observed reality in a relatively rapidly communicable way.

There is a balance between success-at-detailed-explanation and rapid-communicability, and the exact way to strike this balance is going to be subtle and in some cases subjective. But, in the case of religious beliefs, the verdict is quite clear: the religious world view, compared to the scientific world view, fails miserably at explaining the particulars of observed reality in a relatively rapidly communicable way.

The key point here is that, even if people want to stick with their evolutionary-legacy-based inductive biases (which make them intuitively favor superstitious explanations), the failure of religious theories to explain the particulars of observed reality is now so drastic and so obvious, that anyone who really carefully considers the evidence should reject these religious theories anyway.

Maher's film points out sensationalistically silly aspects of religious belief systems. But these aren't really the right anti-religion argument to use, in terms of philosophy of science and the theory of rationality. After all, are the Big Bang and Big Crunch and the evolution of humans from apes really any less everyday-ishly wacky than Judgment Day and the talking snake in the Garden of Eden?

The right argument to use is that, if one assumes Bayes rule plus a Communication Prior (or any other sensible, everyday-reality-based prior), then religious theories fail miserably.

Of course, almost no one on the planet can understand the previous sentence, though ... which is why his approach of dramatically emphasizing the most absurdly wacky religious beliefs and believers is probably a way more effective PR strategy!


The Emotion Prior

Finally, another suggestion I have regarding the popularity of religious beliefs has to do with something my ex-wife said to me once, shortly after her religious conversion to Buddhism, a topic about which we had numerous arguments (some heated, some more rational and interesting, none usefully conclusive nor convincing to either of us). What she said was: "I believe what I need to believe in order to survive."

She didn't just mean "to survive physically" of course ... that was never at issue (except insofar as emotional issues could have threatened her physical survival) ... what she meant was "to survive emotionally" ... to emotionally flourish ...

My (rather uncontroversial) suggestion is that in many cases religious people -- and others -- have a strong bias toward theories that they enjoy believing.

Or in other words: "If believing it feels good, it can't be wrong!"

This is probably the main issue in preaching atheism: one is asking people to

• adopt (some approximant of) Bayes rule with a Communication Prior (or similar)
• actually carefully look at the evidence that would be used in Bayes rule

... rather than to, on the other hand,

• avoid looking at evidence that might disconfirm one's theory
• utilize an Emotion Prior when evaluating various theories that might explain the evidence

The question is then whether, in each individual case,

• the Emotion Prior outweights the Communication Prior (or similar)
• the sociopsychological pressure to look at evidence outweighs the sociopsychological pressure to ignore it

Ignoring evidence gets harder and harder as the Internet broadcasts data to everyone, all the time....

To study these choices in an interesting way, one would need to model the internals of the believer's mind more subtly that has been done in this post so far....

But anyway ... the evidence of the clock in front of me is that I have spent too much time amusing myself by writing this blog post, and now have more useful things to do ... so, till next time!

P.S. Thanks to my wife Izabela for discussions leading to the introduction of the communication-testing variant of the Communication Prior, after the more basic version had already been formulated....

Another Transhumanist Nightmare

Some anonymous freak wrote this story, a piece of transhumanist/absurdist fantasy which includes me in a minor role ... it's childish, but I have to say, mildly amusing...

Another
Transhumanist
Nightmare

The End of the American Era!! (Not)

I'm not generally a very political person ... my thinking and my life-decisions are pretty strongly focused on the "big picture": superhuman AI, the Singularity, transhumanism and all that.

I was deeply into politics as a teen (largely because my parents raised me to be), but as I realized that utopian political dreams were likely to founder on the intrinsic biological perversity of human nature, I drifted away from the political sphere and started thinking more about how to improve or transcend human nature itself....

However, every now and then some piece of political stupidity gets on my nerves sufficiently that I wind up burning time thinking about it.

One of these cases has occurred recently: I've become annoyed by a large number of people proclaiming that "the American era is finally ending." No empire rules forever, and blah blah blah.

I've been hearing this sort of talk for a while, but all the more intensely given the recent week's American banking crisis.

So I decided to write a blog post to get my thoughts on the topic out of my head!

I've never been noted for my patriotism: I really don't care, at a fundamental level, about nations or other related manifestations of contemporary human society. I'll be happy to see them all go away once human nature is fundamentally reformed via radical technological advances.

I've also spent enough time living and traveling outside the US, to get some feel for the strengths and weaknesses of the good/bad old US of A.

My considered opinion of the "end of the American era" meme is that it's pretty much bullshit.

I also seem to look at the current financial crisis a little differently than most others (big surprise there, huh?).

The issues that investment banks, insurance companies and related institutions have recently experienced have been widely attributed to greed, poor government regulation, and so forth. These attributions are surely correct -- but any real event has multiple causes ("cause" being essentially a creature of subjective theory rather than physical reality anyway). And one cause is not being commented on enough, which is the phenomenal practical creativity involved in all the recondite financial instruments (credit default swaps, mortgage strips and the like) underlying the recent woes.

There is some really cool math underlying these financial devices, and this math was largely invented and pragmaticized by American entrepreneurial thinkers. American quants have developed many new fields of financial math, and brought these into the real world, thus moving the global economy to a whole new level of complexity and efficiency.

Innovation always carries risks ... and we've seen that in the markets over the last weeks and months. But let's not forget how amazing the innovations are, and what tremendous positive potential they have.

I agree that exotic derivatives should be regulated more carefully. On the other hand, I also agree with their advocates that they add significant efficiency to the financial markets, and hence are a major asset to the world economy.

Of course, one can theoretically envision socioeconomic systems in which efficiency would be achieved by other, less perverted and convoluted means. But, as history shows, theoretically-envisioned socioeconomic systems are difficult to translate into realities, because of the subtleties of human psychology and culture.

And it's precisely these "subtleties of psychology and culture" that led America to invent quantitative finance ... and so many other amazing technological and scientific developments ... which is exactly why I tend to doubt the "American era" is at its end.

My contention, and it's not a terribly original one (but I may have a somewhat original slant), is that compared to other countries on the planet right now, the USA has a combination of cultural psychology and socioeconomic institutions that is uniquely well-suited to fostering practical creativity.

Note the compound of terms: "practical" and "creativity."

I don't think the US has any kind of monopoly on creativity itself. There are brilliant, creative minds everywhere. Some cultures foster creativity more than others ... and the US is pretty good at this ... but I'm not sure it's uniquely good.

And I don't think the US has any kind of monopoly on practicality, either. Although historically this has been a US characteristic, there are surely other nations that are currently more down-to-earth and practical than the US (as a generalization across various aspects of life).

However, the US seems to be uniquely good at taking creative new ideas and finding the first ways to give them practical implementations -- an art that requires a great deal of creativity in itself, of course.

What is it about the US that fosters practical creativity? It's no one thing. It's a synergetic combination of culture and institutions. The institutions help keep the culture alive, and the culture helps keep the institutions alive. Practical creativity is something that pervades many aspects of US life -- government, research, education and industry, for example. Precisely because of its pervasive and systemic nature, the memeplex that constitutes practical creativity in the US is difficult for other nations to copy, even if they have a genuine desire to.

To see what I mean more concretely, think about three examples: the Internet, the Human Genome Project, and the personal computer. How did these come about?

The history of the PC embodies many classic stories of American entrepreneurism, including the creation of Apple and Microsoft by young nerdy entrepreneurs out of nowhere. But it also tells you something about the flexibility of large US corporations relative to similar institutions elsewhere: it was IBM striking a deal with Bill Gates, some young nerd from nowhere with no real business experience, that set the PC industry on its modern path. Not to mention the freewheeling US corporate research lab culture of the time (Xerox PARC and all that). And the government research funding establishment played its role behind the scenes, for instance in funding the creation of mainframes that Bill Gates played with (often breaking the rules to do so) in high school and college, before starting Microsoft....

The Internet began as a project of ARPA (now DARPA), a US government research funding agency that has its strengths and weaknesses, but is notable for its chaotic approach to funding. DARPA program managers cycle in and out every 4 years so that no individual has too much power over resource allocation decisions. There are certainly "old boy networks" involved, and I've personally been fairly unhappy with DARPA's funding choices in my own research field of AI. However, it's interesting to compare the DARPA funding approach with the approach of, say, the Japanese government. Historically, the Japanese have had a tendency to fund huge, comprehensive, nationwide research programmes: e.g. the Fifth Generation computing initiative (which funded a large number of researchers to work on logic-based AI), or the current focus on robotics technology. As a crude approximation, it seems the Japanese funding system tends to push researchers to "all work on the same sort of thing at the same time", whereas the American research funding system is more chaotic, leading to a greater diversity of ideas getting explored simultaneously. We still are overly trend-following and narrow-focused in the US, from my point of view: for instance, AI funding has focused on narrow-AI, logic-based systems and neural net systems for far too long; and the biology community is taking way too long to wake up to the importance of systems biology. But, compared to the rest of the world, the US research funding system is a hotbed of creative chaos.

And then, once the Internet escaped the clutches of ARPA (due to the legislative action of folks like Al Gore, who famously bragged he "invented the Internet" due to his role in this political process), it spread through the collective activity of masses of software entrepreneurs. The Web was initially developed in Europe, but what made it a huge phenomenon was American entrepreneurship, pushed on by the relative ease of securing angel and venture funding in the US. I lived in Australia in the late 1990's but when I wanted to start a software business I had to return to the US because it was so hard to secure investment for an oddball software startup anywhere else (not that it was easy in the US, but it was a bit less painfully difficult...).

The Human Genome Project (which has ushered in a completely new era of genetics and medical research) was began as a US government initiative, involving a network of university labs. And note that the US graduate education system is still by far the best in the world. Our elementary and high schools are generally pathetic compared to those of other developed nations, though there are many exceptionally good schools out there too (the US being a big, diverse place) ... but by the time one gets to grad school, the US is the place to be. Top undergrads from around the world vie to get into our grad schools, and top PhDs vie for postdoc positions at our universities.

But what accelerated the Human Genome Project was the entry of Celera Genomics into the picture -- a venture-funded entrepreneurial attempt to outdo the government genome sequencing project. The new ideas Celera introduced (shotgun sequencing) accelerated the government sequencing project as well, helping the latter to complete ahead of schedule and under budget. (Now Craig Venter, who founded Celera, is involved with a number of projects, some commercial and some nonprofit within government-funded labs ... including a far-out attempt to create the first artificial genome.)

In each of these three cases -- and I could have chosen many others -- we see a complex combination of individual scientific and entrepreneurial initiative, and the spontaneously coordinated, somewhat chaotic and happenstance interaction of government, commercial and educational institutions. This combination isn't planned in detail, and doesn't always make sense, and makes a lot of really stupid decisions (such as not funding advanced AI research much more amply), but it also does a lot of smart things ... and it interpenetrates with subtle, hard-to-describe aspects of American culture in ways that no one has yet been able to document.

Part of the story, of course, is the incredible diversity of the American population: our scientists and engineers, especially, come from all over the world ... and increasingly our business leaders do too. So American culture isn't exactly American culture: it's really world culture, but with an American slant. And this is one among many major differences between America and other contemporary nations, which is closely linked to the "practical creativity" memeplex. I can't see anywhere in Asia, or anywhere in Europe (except possibly England), adopting the "melting pot" aspect of American culture ... but without this melting-pot aspect, it seems to me that practical creativity will have a lot harder time really flourishing. The diversity of ideas and approaches that comes from welcoming and then chaotically blending cultures and outlooks from all over the world, is a major source of practical creativity.

The move from a manufacturing and service economy to a knowledge economy has become famous. The next step, I suggest, is going to be a gradual shift from a knowledge economy to a creativity economy. As knowledge work becomes commoditized, the really precious thing will be creativity work: but not abstract creativity-work detached from the everyday world ... practical-creativity work, aimed at moving the real world forward in unexpected directions. Because of this, I suspect the US will maintain its cultural and economic leadership role in the world for quite some time.

And we'd damn well better, because with all the debt we're racking up, we're basically placing a huge BET that we're going to dramatically increase our productivity via technological efficiency improvements of various sorts. It's a fairly large gamble, but calculated risks are part of the American way ... as recent events on Wall Street show, this approach definitely has its risks ... but my guess is that this gamble will ultimately pan out just fine.

Getting back to my futurist preoccupations: My best guess is that the bulk of the work of creating the Singularity is going to be centered in America. This work will surely be international -- my own current work on advanced AGI technology involves a team with members in South America, Europe, Australia, New Zealand and Asia as well as the US (no Antarcticans yet...). But there's a reason my company Novamente LLC is centered in the US and not these other countries, beyond historical happenstance ... the US is the place where businesses and nonprofit agencies are most willing to seriously consider the practical value of way-out-there technologies. So long as this doesn't change, the American era is going to keep on rolling ... at least that's my best guess at the moment ...

AGI Intelligence Testing

I spent a while this weekend thinking about what might be the right approach for testing the intelligence of early-stage AGI systems that are aimed at human-level, roughly human-like general intelligence (either as an end goal or an intermediate developmental milestone).

Some of my thoughts are summed up in an essay I posted at

http://goertzel.org/agiq.pdf

I’ll quote the first few paragraphs here:

One of the many difficult issues arising in the course of research on human-level AGI is that of “evaluation and metrics” – i.e., AGI intelligence testing.

It’s not so hard to tell when you’ve achieved human-level AGI — though there is some subtlety here, which I’ll discuss below. However, assessing the quality of incremental progress toward human-level AGI is a much subtler matter. In this essay I’ll present some thoughts on this issue, culminating in a couple specific proposals:

1) Online School Tests, in which AGIs are tested via their ability to succeed in existing online educational fora

2) of more immediate interest, a series of tests called the AGI Preschool Tests (AIP Tests, for short, pronounced “ape tests”), based on the notion of “multiple intelligences” and also on some novel ideas regarding learning-based intelligence testing.

The AIP Tests suggested here are specifically intended for AGI systems that control agents embodied in 3D worlds resembling the everyday human world, via either physical robots or virtually embodied agents. Very differently embodied AGI systems (e.g. systems to be initially taught purely via text without any simulated human-like or animal-like body) would potentially need qualitatively different testing methdologies.

On the Preservation of Goals in Self-Modifying AI Systems

I wrote down some speculative musings on the preservations of goals in self-modifying AI systems, a couple weeks back; you can find them here:

http://www.goertzel.org/papers/PreservationOfGoals.pdf

The basic issue is: what can you do to help mitigate against the problem of "goal drift", wherein an AGI system starts out with a certain top-level goal governing its behavior, but then gradually modifies its own code in various ways, and ultimately -- through inadvertent consequences of the code revisions -- winds up drifting into having different goals than it started with. I certainly didn't answer the question but I came up with some new ways of thinking about the problem, and formalizing the problem, that I think might be interesting....

While the language of math is used in the paper, don't be fooled into thinking I've proved anything there ... the paper just contains speculative ideas without any real proof, just as surely as if they were formulated in words without any equations. I just find that math is sometimes the clearest way to say what I'm thinking, even if I haven't come close to proving the correctness of what I'm thinking yet...

An abstract of the speculative paper is:

Toward an Understanding of the Preservation of Goals
in Self-Modifying Cognitive Systems


Ben Goertzel

A new approach to thinking about the problem of “preservation of AI goal systems under repeated self-modification” (or, more compactly, “goal drift”) is presented, based on representing self-referential goals using hypersets and multi-objective optimization, and understanding self-modification of goals in terms of repeated iteration of mappings. The potential applicability of results from the theory of iterated random functions is discussed. Some heuristic conclusions are proposed regarding what kinds of concrete real-world objectives may best lend themselves to preservation under repeated self-modification. While the analysis presented is semi-rigorous at best, and highly preliminary, it does intuitively suggest that important humanly-desirable AI goals might plausibly be preserved under repeated self-modification. The practical severity of the problem of goal drift remains unresolved, but a set of conceptual and mathematical tools are proposed which may be useful for more thoroughly addressing the problem.

Playing Around with the Logic of Play

On the AGI email list recently, someone asked about the possible importance of creating AGI systems capable of playing.

Like many other qualities of mind, I believe that the interest in and capability for playing is something that should emerge from an AGI system rather than being explicitly programmed-in.

It may be that some bias toward play could be productively encoded in an AGI system ... I'm still not sure of this.

But anyway, in the email list discussion I formulated what seemed to be a simple and clear characterization of the "play" concept in terms of uncertain logical inference ... which I'll recount here (cleaned up a slight bit for blog-ification).

And then at the end of the blog post I'll give some further ideas which have the benefit of making play seem a bit more radical in nature ... and, well, more playful ...

Fun ideas to play with, at any rate 8-D

My suggestion is that play emerges (... as a consequence of other general cognitive processes...) in any sufficiently generally-intelligent system that is faced with goals that are very difficult for it .

If an intelligent system has a goal G which is time-consuming or difficult to achieve ... it may then synthesize another goal G1 which is easier to achieve

We then have the uncertain syllogism


Achieving G implies reward

G1 is similar to G

|-

Achieving G1 implies reward

(which in my Probabilistic Logic Networks framework would be most naturally modeled as an "intensional implication.)

As links between goal-achievement and reward are to some extent modified by uncertain inference (or analogous process, implemented e.g. in neural nets), we thus have the emergence of "play" ... in cases where G1 is much easier to achieve than G ...

Of course, if working toward G1 is actually good practice for working toward G, this may give the intelligent system (if it's smart and mature enough to strategize) or evolution impetus to create additional bias toward the pursuit of G1

In this view, play is a quite general structural phenomenon ... and the play that human kids do with blocks and sticks and so forth is a special case, oriented toward ultimate goals G involving physical manipulation

And the knack in gaining anything from play (for the goals that originally inspired the play) is in appropriate similarity-assessment ... i.e. in measuring similarity between G and G1 in such a way that achieving G1 actually teaches things useful for achieving G.

But of course, play often has indirect benefits and assists with goals other than the ones that originally inspired it ... and, due to its often stochastic, exploratory nature it can also have an effect of goal drift ... of causing the mind's top-level goals to change over time ... (hold that thought in mind, I'll return to it a little later in this blog post...)

The key to the above syllogism seems to be similarity-assessment. Examples of the kind of similarity I'm thinking of:

• The analogy btw chess or go and military strategy
• The analogy btw "roughhousing" and actual fighting

In logical terms, these are intensional rather than extensional similarities

So for any goal-achieving system that has long-term goals which it can't currently effectively work directly toward, play may be an effective strategy...

In this view, we don't really need to design an AI system with play in mind. Rather, if it can explicitly or implicitly carry out the above inference, concept-creation and subgoaling processes, play should emerge from its interaction w/ the world...

Note that in this view play has nothing intrinsically to do with having a body. An AGI concerned solely with mathematical theorem proving would also be able to play...

Another interesting thing to keep in mind when discussing play is subgoal alienation

When G1 arises as a subgoal of G, nevertheless, it may happen that G1 survives as a goal even if G disappears; or that G1 remains important even if G loses importance. One may wish to design AGI systems to minimize this phenomenon, but it certainly occurs strongly in humans.

Play, in some cases, may be an example of this. We may retain the desire to play games that originated as practice for G, even though we have no interest in G anymore.

And, subgoal alienation may occur on the evolutionary as well as the individual level: an organism may retain interest in kinds of play that resemble its evolutionary predecessors' serious goals, but not its own!

Bob may have a strong desire to play with his puppy ... a desire whose roots were surely encoded in his genome due to the evolutionary value in having organisms like to play with their own offspring and those of their kin ... yet, Bob may have no desire to have kids himself ... and may in fact be sterile, dislike children, and never do anything useful-to-himself that is remotely similar to his puppy-playing obsession.... In this case, Bob's "purely playful" desire to play with his puppy is a result of subgoal alienation on the evolutionary level. On the other hand, it may also help fulfill other goals of his, such as relaxation and the need for physical exercise.

This may seem a boring, cold, clinical diagnosis of something as unserious and silly as playing. For sure, when I'm playing (with my kids ... or my puppy! ... or myself ... er ... wait a minute, that doesn't work in modern English idiom ;-p) I'm not thinking about subgoal alienation and inference and all that.

But, when I'm engaged in the act of sexual intercourse, I'm not usually thinking about reproduction either ... and of course we have another major case of evolution-level and individual-level subgoal alienation right there....

In fact, writing blog entries like this one is largely a very dry sort of playing! ... which helps, I think, to keep my mind in practice for more serious and difficult sorts of mental exercise ... yet even if it has this origin and purpose in a larger sense, in the moment the activity seems to be its own justification!

Still, I have to come back to the tendency of play to give rise to goal drift ... this is an interesting twist that apparently relates to the wildness and spontaneity that exists in much playing. Yes, most particular forms of play do seem to arise via the syllogism I've given above. Yet, because it involves activities that originate as simulacra of goals that go BEYOND what the mind can currently do, play also seems to have an innate capability to drive the mind BEYOND its accustomed limits ... in a way that often transcends the goal G that the play-goal G1 was designed to emulate....

This brings up the topic of meta-goals: goals that have to do explicitly with goal-system maintenance and evolution. It seems that playing is in fact a meta-goal, quite separately from the fact of each instance of playing generally involving an imitation of some other specific real-life goal. Playing is a meta-goal that should be valued by organisms that value growth and spontaneity ... including growth of their goal systems in unpredictable, adaptive ways....

w0000000t!!!!

Machine Consciousness (report from the Nokia Workshop)

I just got finished with the two-day Workshop on Machine Consciousness that Pentti Haikonen organized at Nokia Research, in Helsinki.

I probably wouldn't have come to Finland just for this gathering, but it happened I was really curious to meet the people at RealXTend, the Finnish open-source-virtual-worlds team Novamente has been collaborating with (with an aim toward putting our virtual pets in RealXTend) ... so the workshop plus RealXTend was enough to get me on a plane to Helsinki (with a side trip to Oulu where RealXTend is located).

This blog post quasi-randomly summarizes a few of my many reactions to the workshop....

Many of the talks were interesting, but as occurs at many conferences, the chats in the coffee and meal breaks were really the most rewarding part for me...

I had not met Haikonen personally before, though I'd read his books; and I also met a lot of other interesting people, both Finnish and international....

I had particularly worthwhile chats with a guy named Harri Valpola, a Finnish computational neuroscience researcher who is also co-founder of an AI company initially focused on innovative neural-net approaches to industrial robotics.

Harri Valpola is the first person I've talked to who seems to have originally conceived a variant of my theory of how brains may represent and generate abstract knowledge (such as is represented in predicate logic using variables and quantifiers). In brief my theory is that the brain can re-code a neural subnetwork N so that the connection-structure of N serves as input to some other subnetwork M. This lets the brain construct "higher order functions" as used in combinatory logic or Haskell, which pose an equivalent mathematical alternative to traditional predicate logic formulations. Harri's ideas did not seem exactly identical to this, but he did have the basic idea that neural nets can generate abstraction via having subnets take-as-input aspects of the connection structure of other nets.

Once again I was struck by the way different people, from totally different approaches, may arrive at parallel ideas. I arrived at these particular ideas via combinatory logic and then sought a neuroscience analogue to combinatory logic's higher-order-functions, whereas Harri arrived at them via a more straightforward neuroscience route. So our approaches have different flavors and suggest different research directions ... but ultimately they may well contribute the same core idea.

I don't have time to write summaries of the various talks I saw or conversations I had, so I'll just convey a few general impressions of the state of "machine consciousness" research that I got while at the conference.

First of all, I'm grateful to Pentti Haikonen for organizing the workshop -- and I'm pleased to see that the notion of working on building conscious, intelligent machines, in the near term, has become so mainstream. Haikonen is a researcher at a major industry research lab, and he's explicitly saying that if the ideas in his recent book Conscious Robots are implemented, the result will be a conscious intelligent robot. Nokia does not seem to have placed a tremendous amount of resources behind this conscious-robot research program at present, but at least they are taking it seriously, rather than adopting the skeptical attitude one might expect from talking to the average member of the AAAI. (My own view is that Haikonen's architecture lacks many ingredients needed to achieve human-level AGI, but could quite possibly produce a conscious animal-level intelligence, which would certainly be a very fascinating thing....)

The speakers were a mix of people working on building AI systems aimed at artificial consciousness, and philosophers investigating the nature of consciousness in a theoretical way. A few individuals with neuroscience background were present, and there was a lot of talk about brains, but the vast majority of speakers and participants were from the computer science, engineering or philosophy worlds, not brain science. The participants were a mix of international speakers, local Finns with an interest in the topic (largely from local universities), and Nokia Research staff (some working in AI-related areas, some with other professional foci but a general interest in machine consciousness).

Regarding the philosophy of consciousness, I didn't feel any really new ground was broken at the workshops, though many of the discussants were insighful. As a generalization, there was a divide betwen participants who felt that essentially any machine with a functioning perception-action-control loop was conscious, versus those who felt that a higher level of self-reflection was necessary.

My own presentation from the workshop is here ... most of it is cut and pasted from prior presentations on AGI but the first 10 slides are so are new and discuss the philosophy of consciousness specifically (covering content previously given in my book The Hidden Pattern and various blog posts). I talked for half an hour and spent the first half on philosophy of consciousness, and the second half on AGI stuff.

I was the only vocal panpsychist at the workshop ... i.e. the only one maintaining that everything is conscious, and that it makes more sense to think of the physical world as a special case of consciousness (Peirce's "Mind is matter hide-bound with habit") than to think of consciousness as a special case of the physical world. However, one Finnish philosopher in the audience came up to me during a coffee break and told me he thought my perspective made sense, and that he was happy to see some diversity of perspective at the workshop (i.e. to see a panpsychist there alongside all the hard-core empiricists of various stripes).

My view on consciousness is that raw consciousness, Peircean First, is an aspect of everything ... so that in a sense, rocks and numbers are conscious, not just mice and people. However, different types of entities may have qualitatively different kinds of consciousness. For instance, systems that are capable of modeling themselves and intelligently governing their behavior based on their self-models, may have what I call "reflective consciousness." This is what I have tried to model with hypersets, as discussed in my presentation and in a prior blog post.

Another contentious question was whether simple AI systems can display consciousness, or whether there's a minimal level of complexity required for it. My view is that reflective consciousness probably does require a fairly high level of complexity -- and, furthermore, I think it's something that pretty much has to emerge from an AI system through its adaptive learning and world-interaction, rather than being explicitly programmed-in. My guess is that an AI system is going to need a large dynamic knowledge-store and a heck of a lot of experience to be able to usefully infer and deploy a self-model ... whereas, many of the participants in the workshop seemed to think that reflective consciousness could be created in very simple systems, so long as they had the right AI architecture (e.g. a perception-action-control loop).

Since my view is that

• consciousness is an aspect of everything
• enabling the emergence of reflective consciousness is an important part of achieving advanced AGI

my view of machine consciousness as a field is that

• the study of consciousness in general is part of philosophy, or general philosophical psychology
• the study of reflective consciousness is an important part of cognitive science, which AGI designers certainly need to pay attention to
  

One thing we don't know, for example, is which properties of human reflective consciousness emanate from general properties of reflective consciousness itself, and which ones are just particular to the human brain. This sort of fine-grained question didn't get that much time at the workshop, and I sorta wish it had -- but, maybe next year!

As an example, the "7 +/- 2" property of human short-term memory seems to have a very big impact on the qualitative nature of human reflective consciousness ... and I've always wondered the extent to which it represents a fundamental property of STM versus just being a limitation of the brain. It's worth noticing that other mammals have basically the same STM capacity as humans do.

(I once speculated that the size of STM is tied to the octonion algebra (an algebra that I discussed in another, also speculative cog-sci context here), but I'm not really so sure about that ... I imagine that even if there are fundamental restrictions on rapid information processing posed by algebraic facts related to octonions, AI's will have tricky ways of getting around these, so that these fundamental restrictions would be manifested in AI's in quite different ways than via limited STM capacity.)

However, it's hard to ever get to fine-grained points like that in broad public discussions of consciousness ... even among very bright, well-intentioned expert researchers .. because discussion of consciousness seems to bring up even worse contentious, endless, difficult arguments among researchers than the discussion of general intelligence ... in fact consciousness is a rare topic that is even harder to discuss than the Singularity!! This makes consciousness workshops and conferences fun, but also means that they tend to get dominated by disagreements-on-the-basics, rather than in-depth penetration of particular issues.

It's kind of hard for folks who hold different fundamental views on consciousness -- and, in many cases, also very different views on what constitute viable approaches to AGI -- to get into deep, particular, detailed discussions of the relationship between consciousness and particular AI systems!

In June 2009 there will be a consciousness conference in Hong Kong. This should be interesting on the philosophy side -- if I go there, I bet I won't be the only panpsychist ... given the long history of panpsychism in various forms in Oriental philosophy. I had to laugh when one speaker at the workshop got up and stated that, in studying consciousness, he not only didn't have any answers, he didn't know what were the interesting questions. I was tempted to raise my hand and suggest he take a look at Dharmakirti and Dignaga, the medieval Buddhist logicians. Buddhism, among other Oriental traditions of inquiry, has a lot of very refined theory regarding different states of consciousness ... and, while these traditions have probably influenced some modern consciousness studies researchers in various ways (for example my friend Allan Combs, who has sought to bridge dynamical systems theory and Eastern thought), they don't seem to have pervaded the machine-consciousness community very far. (My own work being an exception ... as the theory of mind on which my AI work is based was heavily influenced by Eastern cognitive philosophy, as recounted in The Hidden Pattern.)

I am quite eager to see AI systems like my own Novamente Cognition Engine and OpenCogPrime (and Haikonen's neural net architecture, and others!!) get to the point where we can study the precise dynamics by which reflective consciousness emerges from them. Where we can ask the AI system what it feels or thinks, and see which parts of its mind are active relevant to the items it identifies as part of its reflective consciousness. This, along with advances in brain imaging and allied advances in brain theory, will give us a heck of a lot more insight....

Me on "The Future and You" podcast

I did an interview recently on "The Future and You" podcast ... interested parties may, er, feast their ears at

http://www.thefutureandyou.libsyn.com/index.php?post_id=368080

Topics include AGI, uploading, the nature and ultimate immateriality of the self, continuity of consciousness, and all manner of other expectable rambling blablabla ;-)

In Memoriam Leo Zwell

History is a damn dim candle
over a damn dark abyss
-- W.S. Holt

My grandfather Leo Zwell died last week at age 92, so I thought I'd write a blog post (inadequately) commemorating his existence and lamenting his passing.

Leo and me, 1967

What a really exceptional person he was, and how glad I am to have known him.

He was a crystallographer by profession, and a really outstanding grandfather, but most of all I'll remember him as two things (in no particular order):

• an inquisitive, careful, always-processing, generally-interested mind
• a caring, loving human who always wanted to help, and to see that others were doing well

His attitude toward humanity was a subtle one, alluded to by a phrase I remember from a poem I wrote for him and my grandmother when I was 18 or so: "Cynically and with innocence." Of course, this is a rather typically Eastern-European-Jew sort of almost-but-not-quite-self-contradictory attitude, but he manifested it in a uniquely warm and thoughtful way.

He saw humans as hopelessly flawed, screwed-up animal creatures, dealt a mixed hand by evolution (a frequent saying of his was, "We're really just animals. Considering that we're really just animals like all the other animals, we're really not so bad ... we've actually come a long way.") -- yet he was dedicated to getting the most out of the flawed human-animal mind by understanding the world around him and encouraging others to do the same ... and to helping others nudge their flawed human existences in the direction of satisfaction and growth and cognizance rather than suffering and foolishness....

He thought people sometimes expected too much of humanity, given that after all we're just animals with mildly hyperdeveloped craniums. But then, he was also prone to push people to think more and more, to consider perspectives and avenues beyond what their culture or personality or habitual mind-set presented them with. It wasn't exactly "hope for the best, expect the worst" -- more like "expect the mixed-up and confused, but keep pushing for the better and better."

In conversations and in his own thinking on everyday or scientific topics, he was always willing to approach the world simply, in the manner of a child, just looking at reality without preconceived notions ... yet was also extremely interested in accumulating knowledge, and critical of people who govern their thinking and living in an ignorant way.... He was always great with children (at least until his very last years, when he occasionally became impatient), and one of the things he liked about kids was their natural inclination to be scientists and observers: he was always concerned to carefully and openly observe the world around him (and within him) and see what was actually there.

On a personal level, he was extremely important to me in two specific ways (beyond just being a loving and helpful family member), connected to the two qualities. I mentioned above...

Firstly: He was the one who got me into hard science, in the first place. Both of my parents were into social change and social science and such, and the general milieu of my early youth was left-wing hippie nonconformists, not hard scientists concerned with understanding physical or mathematical reality. Whenever we visited Leo (and we moved near him when I was 6, so I saw him often after that) he lectured me on physics, chemistry, biology, mathematics and what-not ... and he had an endless reserve of stories about experiments he'd done, famous and non-famous scientists he'd worked with, and so forth.

Leo taught me some practical things about science and math, such as tricks for doing mental arithmetic ... and simple trigonometry, Newton's method, the basics of X-ray diffraction and so forth -- but that wasn't really the main point ... I could learn that stuff from books ... the main point was his enthusiasm for science and his immersion in the culture of science ... in the culture of thinking, learning, and communicating with a goal of incrementally understanding more and more about the world.... The cognitive/adaptive/communicational attitude was something he applied to every aspect of existence, including everyday human life, but he saw the scientific sphere as the place where thought and in-depth communication could really flourish.

The other major gift he gave me was: As a child, he was the only example I saw, up-close, of a man who had gotten really deeply into taking care of his kids. My father Ted Goertzel was a very good father and I learned an awful lot from engaging with him in wide-ranging intellectual conversations throughout my childhood ... we also traveled a lot together and played sports together and did many other rewarding things ... but, in our house as a child it was my mom who did the vast bulk of childcare. Leo on the other hand had taken on a large amount of the responsibility for taking care of my mother and her brother, himself -- and I could see this, even as a child myself, in the nature of his relationship with his adult children. I thought this was pretty interesting, and I could see that both he and his children had gained a huge amount from this (at the time, quite unusual) pattern.

Seeing the relationship between Leo and my mom was a lot of what imprinted me with the idea that caring for my kids was something I was supposed to do myself (another thing pushing me in this direction, was probably my mother's own radical-feminist emphasis on male-female equality) ... an idea that continues to absorb a significant portion of my life, as I have ~50% shared-with-my-ex-wife custody of my two still-under-18 kids (my oldest, Zar, is 18 and now a junior in university: damn that makes me feel old!!).

It is impossible for me to estimate the amount of personal reward I've gotten from following Leo down the child-caring path ... or the amount to which my thinking about human and AI cognition has been influenced by carefully observing and partaking of the mental and emotional development of my 3 kids....

These days it is not that shocking for a father to take an intensive hands-on role, but for Leo's generation it was anomalous, and -- while there were some specific situational reasons that pushed him in this direction, such as some temporary health problems on his wife's part -- I'm sure much of the the reason he took on this role as intensively as he did was just his overall passion for being helpful. His kids needed and valued his help, so from his view, it would have been unnatural not to provide as much help as was sensibly possible. He could have left it to his wife, but as a rule, he never really was one to leave things to others (another meme I've adopted from him: I too like to Get Things Done, and have a strong tendency toward getting them done myself rather than relying on others...).

I only knew Leo for the last 41/92 of his life, and I'm not going to try to convey nearly all that I knew of him, but I hope the observations I'll make here will transmit some meaningful (if tiny) fraction of the essence of the person.

This blog post will be sort of disorganized (it already is, and I'm just getting started!) ... I'm going to jump around in time a bit ... but that's the way memory works ... and anyway, as Leo and I discussed a few times, modern physics tells us the directionality of time is projected by the perceiving mind...

The end was distressing but, at least, not marked by great suffering. Leo's body faded gradually during the last few years, with an especially fast decline in the last 6 months after a surgery ... his death itself was about as humane as deaths get, with his daughter (my mother, who did an incomprehensibly great job of caring for him during his last years, all while holding down a very demanding more-than-full-time job running a social service program) at his side. He spent much of his last few hours listening to family members bid him farewell over the phone, and then he curled up in a fetal position and reluctantly let his worn-out body shut down.

One of the most interesting things about the last few years was what an astute observer he was about his own mental and physical deterioration. As each aspect of his functioning declined (arithmetic ability, memory for faces, memory for names, sense of direction), he would note this and analyze the particular nature of the deterioration. And he worried to the end about imposing a burden on his family, to which we would always reply that he had helped us a huge amount in the earlier part of his life, so we were more than happy to return the favor.

Indeed, helping others was -- as I already hinted above -- one of the biggest themes of Leo's life. The receptionist at the old folks' home where he lived for his last few years (Martin's Run) likes to tell a story about the day that Leo felt she looked bored and hungry sitting at her desk by the entrance, so he went to the kitchen and brought her some soup. Walking was slow for him at that time, but, he thought it was imperative that the receptionist be treated well. I don't remember any of the other residents at Martin's Run paying nearly so much attention to the well-being of the hired staff. But this meme went way back in Leo's life, to his roots growing up in Brooklyn during the Great Depression, and then his political involvement in socialist organizations a little later on.

(His interest in socialism was never an abstract or theoretical thing; while, as a scientist, he appreciated very much the value of good theories with explanatory power, he never took much stock in social theories, considering them largely mumbo-jumbo. His interest in socialism was always very practical: he saw the world as full of people who needed help, and he thought that society should be organized in such a way that they got the help they needed. After the truth about the Soviet Union became clearer in the 1970's and 1980's he backed away from the more Marxist variants of socialism, but remained strongly attached to the caring-oriented values at the heart of the democratic socialist tradition.)

The (woefully inadequate) obituary in the Philadelphia Inquirer, after his death, read as follows:

Leo Zwell, 92, a retired scientist with the Joint Committee of X-Ray Powder Diffraction Standards in Swarthmore, died of heart failure on Wednesday at Martin's Run, a retirement community in Media where he had lived since 2002.

Mr. Zwell was a physicist with the Jet Propulsion Laboratory at the California Institute of Technology, U.S. Steel, and the U.S. Bureau of Standards before joining the Swarthmore firm in 1972.

He was a 1934 graduate of Brooklyn College. "He graduated at the age of 19," said his son, Michael, chief executive officer of his own human resources firm in Chicago.

"He was working full time" while in college "to put himself through school and contribute to his family's income," his son said. In the depths of the Depression, there was no money for advanced studies.

The Joint Committee, Michael Zwell said, is a publisher of research about X-raying of materials "to identify what their atomic structure was."

Besides his son, he is survived by his daughter, Carol Goertzel; sisters Gladys Berman and Priscilla Endler; four grandchildren; and a step-grandchild.

Mr. Zwell's wife of 54 years, Etta, died in 1994.

A memorial service will be held at 12:30 p.m. today at Martin's Run, near Route 320 and Paxon Hollow Road outside Media.

A spokeswoman for the Humanity Gifts Registry said Mr. Zwell had donated his body to science.

(Small digressive note: Perhaps the most peculiar thing about this obituary is its failure to mention that my mother Carol Goertzel runs a very successful social service program in the Philadelphia area. Apparently some obituary-writers don't do much homework.)

me and my mom Carol (Leo's daughter),
kickin' it old school back in the good old days ...
1968 or so, Eugene, Oregon,
in the midst of all the hippy madness...
but I was more concerned with my box
than science or revolt or the Vietnam war...

Leo graduated high school at 14 (so he wound up entering college at 15, just as I did ... though he was a younger 15) and went to Brooklyn College, where he majored in chemistry. Unlike me (whew!) he worked full-time while in college, helping out in his father's parking garage (which he found rewarding though at times exhausting ... and one of his favorite topics of discourse was the incredible ethical and intellectual characteristics of his father, Pop Charlie ... Charlie Zwagilski before immigrating to the US from Eastern Europe and getting his name mutated).

Pop Charlie (holding my sister Rebecca)
and Grandma Sarah (holding me)

Pop Charlie before I knew him, back in the day

Not too long after graduating college Leo joined the WWII war effort, stationed in the lab rather than the battlefield, working on various projects related to creating better metallic compounds for use in missiles and other military devices. Like many others of that time, he found his innate tendency to pacifism called into question by the Nazi threat. All or nearly all of his relatives who had not emigrated to the US, were wiped out in one or another anti-Jewish purge in Eastern Europe.

After the war ended, he spent some time as a researcher at the Jet Propulsion Laboratory in California, but eventually had to leave there due to McCarthyism: he was actively involved in efforts to politically organize scientists and engineers, and was hence falsely accused of un-Americanism. This episode had a lasting impact on his psyche, imprinting him with a cynicism about human nature and society that never quite left him after that. But, he went on to a very successful research career at US Steel in Pittsburgh, a major industrial research lab of its time ... and then after "retiring" from US Steel, spent 20 years working at the Joint Committee of X-Ray Powder Diffraction Standards in Swarthmore PA, where he pored over the crystallography research literature and figured out which of the many results published there seemed solid enough to enter into JCPDS's extensive databases.

(I briefly had a job helping him him with some of his JCPDS work during the summer of 1982, after my first year of college, but my duties consisted largely of photocopying and I confess I rapidly quit the job even though the small amount of pay was useful to me ... I did not, and still don't, have the patience to enjoy that sort of work....)

Leo during the JCPDS period

Had he been born a little later (a complicated counterfactual, but let's roll with it for the moment...) he would surely have gotten a PhD; but in his day you could do serious science without one ... and he surely did a lot of it. One trait of his that I did not inherit was his surpassing modesty: though he was brighter and more knowledgeable than many of the more famous and well-recognized scientists in the labs he worked in, he genuinely did not envy their greater glory and external recognition. Rather, he was contented to work in the background, analyzing peoples' data for them, solving the tough problems that no one else could solve, and pushing science ahead by helping others with their work as best he knew how.

Another excellent trait of his that I did not inherit was his incredible carefulness. He would review a body of data again and again, with never-failing concentration and precision, looking for subtle patterns, or subtle clues that some sort of error or irregularity might have occurred. There were various commonly-used formulas in the crystallographic literature that he considered inaccurate because the data from which they were derived had various issues; and I have little doubt he was correct. While I am not cognitively suited to be as careful as he was, watching his approach to understanding data was extremely educational for me, and throughout my career I have sought to work with people who are careful in the same way that he was, knowing that this is a virtue I lack.

He also taught me the value of teams in science. My inclination is to be a lone wolf and seek to range far and wide from others trying to solve the hardest problems by idiosyncratic means -- but he showed me that in science, an awful lot of things can only be achieved via different people with different strengths working together. What I'm doing now in my own career exemplifies this: in my thinking I remain a pretty-far-out-there lone wolf, but I'm happy to be working with a diverse, well-integrated R&D team that probably shares many characteristics with the teams he worked in at US Steel.

One theme that he often returned to in his ruminations and (extensive, sometimes rather excessive!) storytelling was that of Generalists versus Specialists. He considered himself more of a generalist, as he was always seeking to synthesize knowledge from different areas, and look for overall patterns of organization. On the other hand he also had deep specialized knowledge of particular areas such as X-ray diffraction data analysis. Only by integrating generalist and specialist traits, he felt, was it possible to really make profound scientific progress. He saw too many scientists as being generalists or specialists only, and felt that for this reason a lot less progress was made than could otherwise be the case.

In part, it was probably his very humbleness that allowed him to be so helpful to so many other scientists, during the course of his career. As they knew he didn't care to compete with them, they were comfortable sharing their doubts, questions, ideas and hard problems with him. He was always interested in arguing intellectual points -- with anyone, be they a famous scientist or a two year old child -- but rarely was there any rancor involved ... it was very much passionate, abstract argumentation in the Greek tradition. Ideas meant a lot to him, yet it was rare for him to hold someone's bad ideas against them as a human being. He had friends with whom he profoundly disagreed.

I recall, when my sister Rebecca (now a school principal) was 6, Leo was lecturing her on the need to avoid fashionable clothing and such, because making fashion statements was foolish and involved emphasizing the wrong things. (He liked to say how he'd worn the same clothes all his life, and watched them go in and out of style.) Rebecca argued back that being determinedly unfashionable was itself a kind of fashion statement, so should also be avoided according to his principles. He laughed and agreed with her, quite willing to take-lightly his own studied unfashionableness, and complimented her on her thoughtfulness. (Of course, Rebecca ultimately grew up not to be the kind of person who reads fashion magazines....)

One of Leo's and my biggest, long-running arguments regarded the future of humanity.

He believed human nature to be fundamentally flawed, and figured that all attempts to reform society and improve human nature were doomed to fail, based on the fundamentally screwed-up essence of human nature. He saw the scientific process as having a greater perfection than human nature, but still being deeply flawed in various ways due to the underlying flaws in the humans doing the science (when led to, for example, wrong formulas being perpetuated because the individuals advocating them were famous or people were simply too lazy to study the underlying data correctly).

When I argued to him that science could in fact be used to improve human nature, by modifying the brain or by uploading people into computers, he basically laughed off the idea. Not that he felt it was impossible, but that he felt he didn't have the conceptual background to really think about it thoroughly. I really wish I had discussed these topics with him when he was 55 or 60 rather than 75-92, but it just didn't occur to me, mostly because our conversations were more dominated by his own (interesting) interests. The most evocative thing he ever said on this topic was something like: "Fine, Ben, but by the time you modify the human brain to remove all the ethical problems and the foolishness, what you'll have won't be human anymore."

In other words, he saw screwed-up-ness as essential to the nature of humanity ... and he understood himself as inextricably part of this human web of screwed-up-ness ... but nevertheless he felt compelled to devote himself to gaining more and more understanding and helping all the other screwed-up humans as much as possible.

Still, I always had the feeling that if we argued long enough, I might have been able to bring him around to a point of view closer to my own radical futurism. Another of his excellent qualities was his ability to change his ideas and attitudes -- no matter how deep-seated -- based on evidence and reflection. This was evident in his scientific work, and also in his personal life: for instance he was raised with relatively sexist and homophobic attitudes (by modern standards), but gradually revised these over time as he observed they really did not explain what he saw around him, nor accord with his desire for general human happiness.

I also remember arguing with him that it might, one day, be possible to resurrect the dead in a scientific way. My argument was: If quantum theory is correct then all the information about everything that has ever happened is encoded in the perturbations of particles in the universe now, so that in principle dead people could be reconstituted from this information, if a being were smart and powerful enough to collect it and do the appropriate nanoengineering. On the other hand, who knows if quantum theory is correct ... and there are, er, some engineering difficulties in this plan. Leo certainly found this train of thought amusing -- but he was not of the emotional cast to draw any hope from it. As far as he was concerned, once he was dead, he was gone, and that was the end of it ... remote possibilities regarding far-future weirdball engineering feats didn't really enter his emotional world. He did not want to die, but he accepted it, and had no patience for superstitions or wishful thinking.

Another comment he frequently made was that he was "never bored." He claimed not to really understand the concept of boredom. "I always have my own mind," he said. "How can I be bored? There's always so much to think about and to wonder about."

Indeed when your attitudes are "Question Authority!" and furthermore "Question Everything!" (two attitudes that came down to me from both sides of my family with pretty overwhelming force), boredom is hard to come by, because there are always so many things to be questioning....

Even at the end, in fact, when his powers of thought were a fraction of their previous, he STILL was never-bored, and was always thinking and trying to understand things. At the very end, when his memory was gone, he was continually asking questions about the objects in the hospital room: what is that? What is that for? Who put that there?

In his last few hours, right before he lost the energy to talk, he was counting ... he counted from 1 to 36, slowly and carefully, as if to be sure all the numbers were still there, as if by attaching himself to the Platonic realm of numbers he was connecting with a reality more substantial than his fading body and memory ... as a non-religious person, he had no delusions of heavens or hells, but beyond our own personal worries, concerns and attachments, there is always the more permanent and perfect world of the Numbers ...

He had a great love of measurement as a way of understanding the world, and when we emptied out his drawers, boxes and closets after his death, we found a remarkable number of rulers, yardsticks, protractors, compasses, calculators and slide rules. (At a certain point, in Swarthmore, he adopted the habit of asking each visitor to his apartment, as soon as they walked in the door, to name their height ... and then proceeding to measure them, taking his daily dose of kindly schadenfreude from observing how nearly all males tend to overestimate their own heights.) Perhaps his most prized personal possession was his watch: when in the hospital the nurses took his watch from him, he felt completely at a loss until it was returned. I was reminded of the historical theory that the main reason Western civilization advanced so much further than others (such as the Chinese) was the invention, in the middle ages, of quantification, of precise empirical measurement. His career was based on measuring things and recognizing patterns in these measurements, and he was concerned with this till the end.

Well ... having written all that, it still seems pathetically inadequate, and there is so much more to say. Most of all I have left off the very, very long list of people whom Leo and his wife Etta helped in various ways during their lives -- going beyond family and colleagues, comprising a remarkable assemblage of individuals whom they encountered in one random way or another and tried their best to help on their ways through life.

I loved all 4 of my grandparents (the others died some time ago) ... and my other grandfather, Victor Goertzel, was also an accomplished scientist (a psychologist) with whom I had considerable intellectual interaction (for instance, when I was in my early 20's, Victor and my grandmother Mildred and my father and I co-authored a biography of Linus Pauling together) ... but I have to say that Leo is the only grandparent whom I really internalized, to the point of view where I sometimes feel like I have a miniature Leo Zwell homunculus living in some obscure corner of my brain, pointing out to me when someone needs help, and pointing out to me when some point on a data chart is likely to be an outlier, and urging me to doubt all my beliefs and ideas, especially the ones that are most important to me.

One of my favorite phrases was taught to me by my friend Bruce Klein, founder of the Immortality Institute and my collaborator in Novamente LLC: "To abolish the plague of involuntary death."

Indeed: few goals are as important. So sad that it did not happen in time for Grandpa Leo.

He had a good life and a very useful one (not always happy, there were bouts of depression and the usual real-life troubles, but overall a richly rewarding human existence) ... but even at 92 years, I can't help thinking that this excellent person's life was far, far too short.

P.S. The photos included in this blog post are among the many we took from Leo's apartment after his death.

ISO a non-religious foundation for the process of "taking responsibility"

This is a re-post (with light edits) of a post I made last week, that got disappeared due to some IT difficulties regarding poor communication btw blogger.com and my Web hoster.

These were some late-night thoughts about the conceptual logic of morality and responsibility, written to the tune of Charlie Parker's glorious "Au Privave" (hot on the heels of "Step into the Realm" by the Roots, one of the few hip-hop bands I like at all...).

I guess what I am inching toward here is some sort of cognitive theory of moral responsibility ... but I'm really inching there, one teeny little piece at a time.... (Well, some topics lend themselves to speed better than others....)

I'll start with will, and then move on to my main topic of taking personal "moral responsibility" for one's actions.

Don't worry, I haven't turned into a preacher yet (though I haven't shaved for a while and am sporting a fairly spiffy Jesus-like beard, though I've been planning to shave for a few days and just haven't found the time), my basic orientation on these topics is one of systems theory....

So, for starters: Anyone with any sense knows by now that the intuitive feeling of "free will" we have is illusory. Our unconscious decides for us, before any conclusion is derived by the process of conscious ratiocination that feels like it's making a decision.

So why bother with the decision process at all? Why not just go with the flow of the non-ratiocinative unconscious? Because we know that the intensely-conscious decision process helps dynamically restructure our long-term memory in a way that will help our unconscious make better decisions in the future.

Next: Anyone with any sense knows that the notion of "moral responsibility" is, to a large extent, a hanger-on from obsolete religious belief systems.

And, the notion of "taking personal responsibility" for one's actions has -- in most particular instances -- questionable empirical grounding. After all, anything any one of us does, is to a large extent caused by our social and physical context -- as Saddam famously said in the South Park movie: "It's not my fault that I'm so evil ... it's society ... society...." Of course, it really IS society ... that cute little pseudo-Saddam wasn't lying ... and in any particular case, none of us really has the information to tease out the internal from the external causes underlying any of our actions ... but yet, this is a poor perspective to take, in spite of the element of truth underlying it.

(Causality, in the end, is not really a scientific concept anyway: it's a tool that minds use to understand the world. A causes B, from the perspective of mind M, if

• A precedes B
• The probability of events in class B is differentially higher, given the prior and correlated occurrence of events in class A.
• M can fairly confidently analogize that, if it were to carry out some action similar to A, then some event similar to B might be likely to follow

But that's a topic for another blog post, another day ... and is covered to some extent in the last chapter of my co-authored book Probabilistic Logic Networks, which Springer is supposed to publish this month...)

Sooo .... Why bother with the process of "taking moral responsibility" at all? Because we know that doing so helps us structure our long-term memory in a way that will help our unconscious take better actions in the future.

When we do something we wish we hadn't done, the act of assigning "responsibility" to ourselves causes us to insert a "correction signal" into our unconscious, which then modifies the structure of our internal declarative/procedural knowledge base in a way that makes it less likely we'll do similar regret-worthy things in the future. This is the case even though we (i.e. the deliberative, ratiocinative, "decision process" aspect of ourselves) don't know that much (rationally or intuitively) about how the unconscious works, and can't really untangle the various causal threads weaving through our minds and our worlds and leading to our actions.

The "ordinary waking state of consciousness" that most people occupy most of the time, involves a coupling of ratiocinative-decision-making with the free-will illusion, and a coupling of moral-responsibility-taking with some semi-religious notion of moral-agency. But it's possible to get into a state of mind where you carry out ratiocinative-decision-making and moral-responsibility-taking without any significant illusions attached to them ... simply because these cognitive dynamics tend to lead to effective overall system functioning.

Now, when I say "it's possible to get into a state of mind where X holds", am I saying that "by exercising one's free will, one can cause oneself to get into a state of mind where X holds" ?

Not really. What I'm saying is that "Sometimes the self-organizing dynamics of a mind coupled with an environment will result in that mind getting into a state of mind where X holds."

And what's the point of me telling you this? Well, some states of mind want to spread from one mind to another....

The basic idea is: If one does not internally take responsibility for one's own actions, one will never send those necessary correction-signals to one's own unconscious. Then one will just keep on doing those regrettable things.

Removing the obsolete, flawed quasi-religious concepts of blame, shame and so forth from one's inner mental landscape is an important step toward becoming a rational and self-aware, fully-realized person; but, once they are removed, they need to be replaced with something else ... they need to be replaced with a recognition of the mind as a holistic, complex dynamical system; and with a recognition of the role of the deliberative, ratiocinative aspect of mind as modulating the complex nonlinear dynamics of the unconscious.

None of us can control ourselves, none of us is fully aware of the dynamics by which we operate (in part because of basic information-theoretic limitations on the extent to which any finite system can understand itself; in part because of information-theoretically unnecessary limitations posed by the human brain architecture, which did not evolve in situations where acute self-awareness and mental self-control were key aspects of the fitness function). But "we" (the deliberative, ratiocinative "phenomenal self" portions of our minds) can modulate the dynamics of the other portions of our minds, via doing things like rational-decision-processes and responsibility-taking....

Our Lovable Species

I thought you might enjoy this email that I (and some others) received this morning. It is fairly typical of its genre, but particularly amusingly worded. I guess it doesn't need much comment; it kinda speaks for itself.

What scares me a little is that this sort of attitude is probably far more common than the transhumanist/Singularitarian/futurist attitude of most of the people I regularly communicate with ... and would probably be even more common if more people were aware of the kind of work that futurist-minded scientists and engineers are doing!

Clearly we need some more Hollywood flicks in which (preferably really hot-looking) AGI's save the world....


From: ********
Date: Sat, Aug 9, 2008 at 3:23 AM
Subject: I hope you realize.....
To: ****

I hope you people realize the immense stupidity, not intelligence, of what youre undertaking.

Anyone with any common sense would not allow this kind of research to reach its natural conclusion: the destruction of our humanity, such as what the transhumanists want to do.

Implants which are used to communicate across networks, or AI's so powerful that they literally take over normal human responsiblities. AI's would conclude we are incapable of caring for ourselves without help if they got advanced enough...

Take Kevin Warwiks work at the University of Reading. Absolutely monstrous. Everything he does, and everything you people do creates major issues in ethics and bioethics.

Trying to make people better with machines...We already are better then any cyborg you can make.

Anyone with any kind of common sense would reject these sort of things. Keep computers in a sense below where we are at.

Anybody who even thought about this, would say hey isnt there a problem with sticking machines in people, first it will be the disabled at first, sure, then normal people because somehow its cool and then alot of other horrible things...

Our technology is outpacing our wisdom, our humanity, our ability to comprehend the ethical boundaries beyond this, when you start getting into the term super intelligence.

The super intelligent thing would be to shutdown implants that react directly with the human mind, exceptions being people that medically benefit, such as alzheimers, the crippled, etc, but they should never be in perfectly healthy human beings.

If God had intended that machines be a part of us, he would have made us like that. But who knows if you guys even believe in Him.

No matter, just be aware of the dangerous waters you swim in. Our technology is so far ahead of our spirituality, that it represents a singularity of its own. A black hole that will suck us into oblivion unless you take a step back and realize the dangers of all this heady research.

So stop a bit, smell the roses, then reflect on this whole madness, Heres to You Mr Warwick of Reading, God help us all.

Self is to Long-Term Memory as Awareness is to Short-Term Memory

This is a brief addendum to a post I made a while back where I gave a casual but mathematical discussion of will and reflective awareness in terms of self-referential structures ("hypersets").

There, the following recursive definitions are given:

"S is conscious of X" is defined as: The declarative content that {"S is conscious of X" correlates with "X is a pattern in S"}

"S wills X" is defined as: The declarative content that {"S wills X" causally implies "S does X"}

Funky, huh? Chew on that for a while!

My point here is to posit a similar definition for that strange beast called the "phenomenal self" (and for a gloriously, Germanically thorough treatment of this entity, please read Thomas Metzinger's masterwork Being No One):

"X is part of S's self" is defined as: The declarative content that {"X is a part of S's self" correlates with "X is a persistent pattern in S over time"}

One thing that's nice about this definition is the relationship that it applies between self and awareness. In a formula:

Self is to long-term memory as awareness is to short-term memory

Elegant, huh?

Your self is nothing more or less than the awareness of your persistent being.

Your momentary awareness is nothing more or less than the self of your instantaneous being.

(Time-span makes a big difference! Indeed, time is almost equivalent to "difference." But that's a subject for another post, for another late-night fueled by too much green tea and too many weird ideas...)

AGI in Xiamen ... and some rambling on the "creativity economy"

I just returned from 2.5 weeks in the Orient ... a week in Japan, doing biz meetings, going to a virtual worlds conference, seeing an awesome guitarist at a weird bar called BarTube, visiting an old friend, and hanging out w/ my son who is staying there for a month studying Japanese and playing Go ... a day or so in Seoul (visiting a humanoid robotics research group and a virtual pets company) ... a few days in Hong Kong giving a talk at WCCI's Human-Level AI session (on how to make a human-level NLP system by partly cheating, see the paper at http://novamente.net/papers) ... and a week in Xiamen, which is in China right across the water from Taiwan ... here's the beautifully situated Xiamen University ...

My friend Hugo de Garis (inventor of evolvable computing, prophet of the Artilect War and all around creative thinker)

(there's me and Hugo and his wife) ... is now a prof at Xiamen University and he's pulling together a humanoid robotics team, whose goal is to spend 4 years making an intelligent computer brain for a Nao humanoid robot:

Hugo and I have been plotting a way to make a clever Nao via using his evolved neural nets for perception and action, and the OpenCog system for cognition and overall system control. The Xiamen folks seem to like the plan and we're discussing the possibility of me spending a couple months there each summer to collaborate, and them funding some students to work on the OpenCog side of the project. If someone follows ahead with the idea I've been selling, of integrating a robot simulator (like Gazebo) with a virtual world (like OpenSim), this could synergize really nicely with Novamente's AI-in-virtual-worlds stuff....

And Xiamen would be a very nice place to spend summers...

I've been fascinated by China since youth, probably due to my mother doing grad work in Chinese history and philosophy back then. She gave me a bunch of Chinese history books and stories and poetry to read, which made me fascinated with the culture. When I was 17, halfway thru my 3rd year of university, I applied for a scholarship to go to China for a year and do research relating non-well-founded sets (hypersets) to Buddhist cognitive philosophy. But the scholarship required me to know Chinese and I didn't, so I didn't go.... (I've never been able to put much energy into learning languages... too much other interesting stuff to study and think about ... and I find it hard to pick up languages via immersion because of my habit of not paying attention to what anyone is saying or doing around me ... so I'm rarely actually immersed in anything but my own thoughts ;-O ) ...

ANYWAYS ... I met F2F Novamente & OpenCog's Chinese contributors, which was very nice... here we have (back: Lian Ruiting, Guo Junfei, Chen Shuo, Rui Liu, me)

Very smart, interested, ambitious people!

Hugo is convinced that China is the country of the future and America is already obsolete. He foresees a coming century of reverse brain drain, where China recruits smart scientists and engineers from Western nations....

It might happen -- I don't rule it out. Of course, unlike Hugo, I think some sort of technological Singularity is very likely by mid-century and maybe sooner -- but let's ignore that for the moment ... talking just in conventional political/cultural terms, it's not obvious to me that he's right.

No doubt China has very many very smart and ambitious and hardworking people (like the ones pictured above!) ... but the cultural differences w/ the West are profound and I don't think any of us understands what they mean in terms of the future of science and engineering.

One observation I like to make is as follows. People talk about the knowledge economy ... where manual work has long been outsourced to 3rd world countries, leaving 1st world countries increasingly consumed w/ knowledge work.

And more and more so, the US becomes a pragmatic knowledge integration economy -- specialized knowledge like programming and science gets farmed out to 3rd world countries, but the task of integrating together various pieces of knowledge for practical purposes is still done in America. Even in Novamente, which is a damn international company, we do programming and science and project management overseas, but the figuring-out of what programming and science needs to be done to serve business goals, is largely done in the US. Because the US is where our customer companies are -- even if their work is largely done overseas, the high-level staff defining their vision are mostly here. The matching-up of technology and business, where Novamente is concerned, occurs mainly within the arena of US culture. (We do have overseas customers, but they are either run by Americans or following business models that closely copy American ones.)

The next step, I think, is the creativity economy. Even integrative knowledge will become commoditized. Creation of new ideas will be the LAST thing to get commoditized. But this is exactly where America excels. No nation on Earth fosters creativity as well as the USA. And for this reason, I'm not so sure that America's period of dramatic success is over. The more science and technology accelerate, the more critical creativity becomes -- and, lame as American culture and institutions are, they seem better than most alternatives at fostering wide-ranging creativity. (The only cultures I've known that seemed maybe more creativity-friendly were Australia, New Zealand and Hungary. But those are small places, population-wise.)

There is loads of creativity in China, for instance, on a personal level. Very creative people. But I'm not sure the culture fosters creativity in the way that US culture does. Oriental culture seems to favor obedience a lot more than US culture, and creativity is often not compatible with obedience.... The US is probably the most anarchic major developed country -- which has its downsides, especially for those below the poverty line in the US -- but, it seems that anarchy and creativity are inextricably entwined.

If China evolves a culture of creativity, then Hugo will be proved right and this will become the Chinese century ... and maybe the Singularity will get launched in China (hey, maybe it will get launched there anyway via Hugo's and my collaboration!!!)..... But that's a big "if", I suppose. Yet one feature of Chinese history is its tendency toward sudden, radical changes of one sort or another. Time will tell.

Anyway I look forward to returning to Xiamen and other parts of China when my schedule permits (hopefully for a couple months next summer, and a couple weeks in the fall or winter) ... there is a definite energy there that I don't find in developed countries these days, nor in 3rd world countries ... there is a feeling of "waking up" and progress that is exciting...

And, more importantly, there is a possibility of creating a thinking machine and doing other amazing technology projects there more rapidly than in other parts of the world, due to the availability of brilliant scientists and engineers at a relatively low cost (esp. outside the tier 1 cities). Whether or not China develops a culture of creativity allowing it to "own" the next century, there are loads of opportunities for international collaboration ... like what Hugo and I are trying to set up....

But anyway. Enough rambling. I've been sleep-deprived since returning from China, due to jet lag issues ... tonight I'll go to sleep "early" (i.e. maybe by 1AM) and hopefully actually get a full night of sleep.. (yah right...)

Eureeka!! -- The Underlying Logic Unifying Quantum Theory and General Relativity, Revealed Over a Plate of Sour Fish Consumed Over South China

Eureeka!! -- The Underlying Logic Unifying Quantum Theory and General Relativity, Revealed Over a Plate of Sour Fish Consumed Over South China; Plus Long Digressions on Mark Twain, the Pathetic Woes of Middle Age and the Good Old Mongolian Skin-Peeler

[I wrote this post 2 weeks ago, but didn't get around to posting it due to being in China, with a slow Net connection..]

En route from Seoul to Hong Kong, exhausted from a 5 hour night's sleep following a 4 hour night's sleep, over-jazzed by too much strong coffee (which I rarely drink), stomach-sickened by ordering and consuming random dishes in a Korean restaurant via pointing at random hieroglyphs on the menu and hoping vainly for the best ... head full of Mark Twain's wacky biography which I just finished ... irrationally nervous due to having left my oldest son in Japan to tour around on his own for a week before his Japanese class in Kanazawa starts (yes, he's mature enough to handle himself ... and Japan is a damnably safe place, aside from the risk of spending all your money ... but even I the ultimate anarchist parent can't help a bit of worry) ... dulled almost but not quite to a stupor by a relentless series of software technology oriented business meetings (all with wonderful and interesting people, but still, there's only so much meeting I can take) ... I picked up Lee Smolin's book on quantum gravity, which I bought for my physician-cum-maverick-physics-theorist father-in-law years ago but never read myself ... and while reading a totally irrelevant passage and eating the oddly sour fish that passes for food on Korean Air, some very simple and obvious ideas popped into my mind, and I realized to my surprise that, via converging together several streams that have been tumbling through my head for years, I'd happened upon what appeared to be the correct probabilistic logic of unified quantum gravity.

I'm eager to write up this logic in a paper, but, I've promised myself not to undertake anything serious -- except tasks critical for Novamente as a business, or in order to fulfill obligations already incurred -- until the OpenCog Prime (http://www.opencog.org) wiki pages are done (maybe another 20-30 hours of work, but hours for concentrated writing/editing work are very hard to come by these days due to the combination of business obligations and ongoing research projects needing supervision and/or feedback).... But I'll indulge myself in a brief blog post on the topic as a stopgap - partly to ensure the idea doesn't escape from my mind tonight when I finally slip into the deep sleep my body's been craving for 72 hours or so....

Twain's bio was a fascinating read, by the way. Three things among many others struck me, viewing his life-story from a selfish view in terms of its potential lessons for my own life. One is the way he spent a load of his time on stuff other than writing -- business of various sorts, as well as lecturing, traveling and so forth. But these "distractions" didn't seem to detract from his productivity as a writer as much as I would have thought -- they filled his head with stimulation and ideas, and no doubt made his writing more interesting than if he'd just sat home writing all day. Second is the romance he found in business pursuits ... which reminded me a bit of Rimbaud, who gave up poetry as a very young man after too few years as a writer, and wasted his twenties chasing African gold, ultimately dying from poisoning attained via wearing gold under his undies to hide it from thieves. Rimbaud, due to his premature death among other issues, failed to transform his digressive life experiences into art. I can see in my own psychology the excitement that the business world held for these people: it does stimulate parts of the mind that creative art and science don't touch. Finally I'm struck by the amount of real trash literature Twain produced. I'm reminded of Danilo Kis's (a truly great Serbian writer -- thx to Predrag Janicic for waking me up to him) comment that he didn't write his complete works, only his selected works. Twain was not like that. Twain's best work was awesome, his worst work was terrible. He could have omitted a good 50% of his production and his legacy would be greater not less. Philip K. Dick had the same property: there's Ubik ... and then there's Dr. Futurity.... The lesson for me is, I suppose, not to worry too much about spending time on apparently digressive pursuits (like writing this blog post, um) -- so long as they're feeding the creative engine one way or another -- and given the limited time I have for creative pursuits, to try hard to be more like Kis than Twain or Dick, and filter out crappier works before I take the time to produce them.

Another striking thing about Twain was the way he foresaw the power of machinery to alleviate human suffering. A lesson that seems obvious these days but was surprisingly poorly understood in his times, even though the industrial revolution was in full swing and new mechanical inventions of all sorts were pouring out of human minds at an amazing rate. If you haven't read it, his Connecticut Yankee in King Arthur's Court -- arguably the first American SF novel -- is a hilarious and deeply insightful premonition of the promise and peril of advanced technology. As a time travel fable, it's got Back to the Future beat by a long shot, without need of paradoxical absurdities beyond those intrinsic to human nature.

And now... what about quantum gravity...

Three threads need to be drawn together, into a single mathematical formalism. But there really seems no obstacle to doing so. (Except time of course, which is a distressingly rare commodity for me these days. Must confess to a bit of jealousy of my son as he wandered off from Tokyo to Kyoto, with no specific plan for spending his days, other than to amuse himself. I never intended to accumulate so many obligations -- keep companies running, organize conferences, pay other peoples' college tuition, close a mortgage, finish a book, pay child support each month, drive the kids to and from school, blah blah blah blah blah ... I once actually thought I'd live the life of the "free and easy wanderer" from the Chuang Tzu (on my mind as the flight I'm on approaches China), or maybe of Paul Erdos who freeloaded off one friend after another as he spent his life journeying around the world doing mathematics and taking drugs ... I never envisioned taking on all this responsibility for other people (kids, wife, ex-wife) and organizations (companies, non-profits, egads!) ... yet it's all wonderful, interesting stuff ... people and ideas I really care about ... so I'd really be an ass to complain ... it's a fantastic time to be alive ... yet not quite as fantastic as a few decades hence will likely be, when minds will be far more fully liberated from the horrifying/stultifying constraints of legacy human physiology ... but, well, anyway...)

Quantum gravity!

Thread one was invented by Saul Youssef, and I've written about it before. Check out the lovely bibliography he's assembled at 

http://physics.bu.edu/~youssef/quantum/quantum_refs.html

Brilliant, brilliant man. A hero of our time! Someone give that man a muffin!!

The observation here is that if you're willing to take the step of assuming probabilities are complex rather than real numbers, the basic rules of quantum theory fall right out. This is one of those things that seems shocking and weird at first, and then seems tremendously obvious after you read through the math. Three cheers for Saul Youssef!!

Thread two is something I came up with a couple years ago, and wrote up in a paper which I'm in the middle of submitting for publication. I sent the paper to a journal and they sent it back asking me to provide names and mailing addresses of eight referees able to review the paper. I've been lagging on that task along with a huge amount of other stupid paperwork that's accumulated during the last N years. I guess the editor couldn't think of anyone to send it to. The idea, anyway, is infinite order probability.

An ordinary probability is a probability of an event. A probability distribution is a function that assigns a probability to each one of a set of mutually exclusive outcomes of some event (the different values assigned to different exclusive outcomes must sum to one). A second-order probability is a probability distribution over probability distributions ... it's a function that assigns a probability to each one of a set of probability distributions. A third-order probability ... etc.

An infinite-order probability is a function that assigns a probability to each one of a set of infinite-order probability distributions. Sounds odd, but it's a mathematically consistent idea, as I showed in my paper. I also showed that these oddball entities are closely related to some much more familiar and intuitive mathematical entities, Markov matrices.

The third thread is causal networks. A foundational notion in general relativity is causality. The causal network of events, in relativity, tells you for any pair (A, B) of events, which ones have the property that A is causal for B. This has to do with the finitude of the speed of light: if A and B are too close in time and too distant in space, there may be no way for A and B to causally affect each other.

If A and B are not causally related, there may still be some event C so that C is causal for both A and B. In that case we may say that, probabilistically speaking, A and B are independent conditional on C. That is,

P(A & B | C) = P(A | C) P(B |C)

The causal network gives us a set of independence assumptions on the space of events.

General relativity is in essence a dynamic on causal networks: it tells you how a causal network at one time (plus some extra information) gives rise to a different, related causal network at a subsequent time.

Finally let's reflect on what Smolin (see Three Roads to Quantum Gravity) calls the "strong holographic principle." His reasoning for this principle is subtle and involves the Bekenstein bound and related results, which state that all the information about the interior of some physical region, may actually be thought of as being contained on the surface of that regions. (He explains this better than I could, so I'll just refer you to his book.)

What the principle says is: a la Nietzsche, there are only surfaces. Re-read Nietzsche's Twilight of the Idols and you'll see that he presaged quantum gravity here, in a similar way to how he presaged quantum theory proper in his vision (in The Will to Power) of the universe as composed of a dancing swarm of discrete interacting quanta. Kant posited phenomena and noumena, Nietzsche saw only noumena. Smolin also. Smolin views the universe as a collection of surfaces, each one defined as a relationship among other surfaces. Put in words like this, it sounds mystical and fuzzy, but there's math to back it up -- the words just hint at the mathematical reality.

But is each of these Smolin surfaces definitively known? No. Each one is probabilistically known. And if each of these surfaces is to be thought of as a relationship between other surfaces, then this means each of these surfaces is most directly modeled as a hyperset (see my prior blog posts on these mathematical constructs). (This is not how Smolin models things mathematically, but I doubt he'd be opposed, as he's used equally recondite math structures such as topoi.) So these surfaces should be modeled as probabilistic hypersets -- aka infinite-order probability distributions.

But what kinds of probabilities should be involved in these distributions? Clearly, Youssef has taught us, these should be complex probability distros -- or in my variation, infinite-order complex probability distributions.

The inescapable conclusion is: The physical universe is a dynamically evolving causal network defined on an infinite-order complex probability distribution.

You read it first here, folks.. ;-O

Or, to put it a bit more conservatively: A useful, perhaps critical language for modeling quantum gravity phenomena is the logic of causal networks on infinite-order complex probability distributions.

There are fun connections here with the psychology of self-awareness and free will, as I've discussed in a couple previous blog posts (follow the links). According to those blog posts, a good way to model reflective awareness would be using infinite-order real probability distributions; and a good way to model will would be using causal networks on these distributions. What quantum theory introduces is the complex-number probability aspect, which makes everything counterintuitive and weird.

I hope I can really find time -- amidst the manifold obligations of middle age plus the not incidental life-task of creating superhuman AI, plus other distractions like bioinformatics and fiction and music and what-not ... and family and the occasional personal entertainment -- to write these ideas up carefully, because I really do think they have deep potential.

There seem to be more connections lurking here: the logic of causal networks seems somehow inescapably tied up with Clifford algebras, providing a tie-in with my algebra of multi-boundary forms (my only publication in a physics journal so far, but it's really a math paper). Presumably one can go from causality somehow straight to discrete Clifford algebras using some kind of axiomatic derivation, and from there to the various beautiful algebraic symmetries underlying modern physics ... Gell-Mann's "Eightfold Path" and its kin ... but anyway, the flight's about to land and the stewardess wants me to put away my laptop, so the blog post is gonna end .. I'll post it online when I get back to the hotel assuming there's functional internet there ... the inimitable Yan King Yin (famous on various AI email lists) is picking me up at the airport and I'm curious to meet him, although I'm so worn out I'm not sure I'll be lucid enough to milk the occasion's potential for lively AI discussions....

How about the bloody Yverse (see previous blog post on this)? Each Smolin surface ... each relation in the network of interdefining hyperrelations ... defines its own multiverse: a quantum multiverse relative to its own perspective. The network of surfaces (aka relationships) is then a Yverse. QED.

Another day, another dozen digressions ... it's SOOOOO tempting to take a few days and formalize the logic of causal nets over complex infinite-order distros, but instead (inbetween biz meetings and AI research meetings and conference speeches and meals with AI colleagues) I'll spend my "spare" hours in the next few weeks on the OpenCogPrime documents ... a very tedious matter of taking about 50 wiki pages from the Novamente wiki site and editing them down into OpenCogPrime rather than Novamente Cognition Engine pages... yecch...

(Any wealthy patrons out there want to hire me a secretary, a housecleaner and a scientific assistant? I can't promise the Singularity will be accelerated by a few years but it's a definite possibility. For sure a lot more fascinating math, art and science would be generated were I to be thus endowed. (And, getting back to Twain, I wonder what additional great works he would have produced if some of his rich friends had decided to fund him, sparing him the financial anxiety that led him to waste years of his life on various harebrained business schemes. Yeah, they provided grist for his creative mill ... but there's such a thing as too much grist and not enough time to mill.) But I can't complain too much (er, OK, wait, I guess I am...) ... whenever I get TOO frustrated at the realization that 50% of my really good ideas and creations will remain forever unarticulated or un-worked-out-in-detail because I've failed to be born rich or become rich (so far), I remind myself of my favorite Haruki Murakami character, the Mongolian Skin-Peeler ... a World War II torture artist who tortured Chinese prisoners of war by slowly peeling their skin off ... as I see Ulaanbaator on the video screen of the plane as it approaches Hong Kong (haha, I'm a bad boy and failed to shut off my laptop when instructed ... how very non-Oriental of me!!) it's hard for me not to feel thankful that I'm not one of his victims ... I've got my epidermis attached to my dermis, woo hoo! ... and I at least have time to work out a nontrivial percentage of the cool ideas and creations that course through my overheated brain...)

(While you're at it, imaginary patron, recruiting Novamente LLC a CEO with lots of game or virtual world industry experience would be nice. I think I'm doing a decent job as CEO, with help from Bruce and Cassio and Wendy and my other wonderful colleagues, but it would be nice to have a sufficiently complete management team that I could spend 80% of my Novamente-time on science rather than business. And you may as well recruit us a kick-ass project manager too, so Cassio can help me out with research and retire from project management. (Ok, dream on, Ben.... And remember the Mongolian Skin-Peeler....). And while you're at it, throw in maybe $1M per year so that I can actually fund a team of kick-ass programmers to build a thinking machine ... in case you haven't heard I have a pretty detailed and well-argued design for one, but it's getting built bloody fucking slowly due to lack of funding, and because it's not the sort of thing where partial progress yields exciting incremental results, any more than building 30% of a human brain would yield a 30% functional human... but I dididididididigress ;_)

(I think my wife is really, really tired of hearing about the Mongolian Skin-Peeler. He seems to occupy an unjustifiably prominent role in my emotional topography. Read The Wind-Up Bird Chronicle.)

Mark Twain, I add, would have been a hell of a blogger; far more entertaining than me. He wrote a dozen letters each day back then in the pre-digital dark ages.

Time to get off the plane.

Open-Source Robots + Robot Simulators + Virtual Worlds + AI = ???

I’ve been reading up on the iCub open-source humanoid robot lately, and I think it’s pretty exciting. Given what open source has done for Web browsers, bioinformatics tools and other sorts of software, the possibility of harnessing the same development methodology for robot hardware and software development seems almost irresistably exciting.

I’m no roboticist, but I do know something about the AI software that robots need to understand the world and act in it – and I’ve been doing a lot of work lately on the use of AI to control simulated agents in virtual worlds. In this vein, this blog entry contains some follow-up thoughts about the possibility of building connections between the iCub and various other relevant open-source software systems relevant to AI and virtual worlds.

For starters: What if someone made a detailed simulation of iCub in Gazebo, an open-source 3D robot simulation platform? Then folks around the world could experiment with iCub without even building a robot, simply via writing software and experimenting with the simulation. Experiments with other robots and Gazebo have shown that the simulation generally agrees very closely with real-world robotic experience.

And what if someone integrated Gazebo with OpenSim, the up-and-coming open-source virtual-world platform (which uses an improved version of Second Life’s user interface, but features a more sophisticatedly architected and flexible back end, and best of all it’s free)?

Furthermore, work is underway to integrate OpenSim with OpenCog, an open-source AI platform aimed at advanced machine cognition (yes, I’m one of the organizers of OpenCog); and OpenSim could similarly be integrated with OpenCyc, OpenNARS, and a host of other existing open-source AI platforms. Throngs of diversely customized, simulated iCubs controlled by various AI algorithms could mill around OpenSim, interacting with human-controlled avatars in the simulated world, learning and sharing their knowledge with each other. The behaviors and knowledge learned by the robots in the virtual world could then be transferred immediately back to their physically embodied brethren.

What stands between us and this vision is “just” some software integration work ... but of course, this kind of work isn’t easy and takes time and expertise. For various economic and cultural reasons, this sort of work has not been favored by any of the world’s major R&D funding sources – but the open-source approach seems to have increasingly high odds of getting it done. It seems at least plausible that iCub won’t go the way of OpenPINO and other prior attempts at open-source robotics, and will instead combine with other open-source initiatives to form a key part of a broadly-accepted, dynamically evolving platform for exploring physical and virtual humanoid robotics.

Artificial Wisdom (... episodic memory, general intelligence, the Tao of John Coltrane, and so forth)

Every now and then, someone suggests to me that, alongside the pursuit of Artificial Intelligence, we should also be pursuing "Artificial Wisdom."

I always figured the "artificial wisdom" idea was probably just a bunch of useless English-language wordplay -- but one night last week, while watching Idiocracy with the kids for the second time (great movie exploring a non-Singularity-based future by the way ... highly recommend it!), I spent a while surfing the Web on my laptop refreshing my memory on how others have construed the "wisdom" concept and musing on what it might mean for AI.

Surprisingly enough, this led in some moderately interesting directions -- nothing revolutionary, but enough to justify the couple hours spent musing about it (and another 90 minutes or so synthesizing and writing up my glorious conclusions).

My main conclusion was a perspective in which wisdom is viewed as one of three core aspects of intelligence, associated with three distinct types of memory:

• cleverness, associated with declarative memory (and the ability to manipulate abstract, certain or uncertain declarative knowledge)
• skillfulness, associated with procedural memory (and the ability to effectively learn and adapt new procedures based on experience)
• wisdom, associated with episodic memory (and insightful drawing of large-scale conclusions therefrom)

This being a blog post, though, rather than just presenting my conclusion, I'll start out by recounting some of the winding and mostly irrelevant path that led me there ;-)

Classical Conceptions of Wisdom

I started out with the dictionary, and as usual found it close to useless....

A typical dictionary definition of "wisdom," which is not a heck of a lot of help, is from Wiktionary, which tells us that

wisdom (plural wisdoms)

means

1. An element of personal character that enables one to distinguish the wise from the unwise.
2. A piece of wise advice.
3. The discretionary use of knowledge for the greatest good.
4. The ability to apply relevant knowledge in an insightful way, especially to different situations from that in which the knowledge was gained.
5. The ability to make a decision based on the combination of knowledge, experience, and intuitive understanding.
6. (theology) The ability to know and apply spiritual truths.

and furthermore that

wise

means

Showing good judgement or the benefit of experience.

Hoo haw.

These definitions don't give us any particularly interesting way of distinguishing "wisdom" from "intelligence." Essentially they define wisdom as either intelligence, spiritual insight, or the application of intelligence for ethical ends. Nothing new here.

Wikipedia is slightly more useful (but only slightly). Firstly it notes that

A standard philosophical, (philos-sophia: literally "lover of wisdom"), definition says that wisdom consists of making the best use of available knowledge.

It then notes some psychological research demonstrating that in popular culture, wisdom is considered as different from intelligence. Psychological researchers are quoted as saying that though "there is an overlap of the implicit theory of wisdom with intelligence, perceptiveness, spirituality and shrewdness, it is evident that wisdom is a distinct term and not a composite of other terms."

More interestingly, Wikipedia notes, Erik Erikson and other psychologists have argued that it is, in large part, the imminence of death that gives older human beings wisdom.

The knowledge of imminent death is seen as focusing the mind on concerns beyond its own individual well-being and survival, thus inducing a broader scope of understanding and an identification with the world at large, which are associated with the concept of wisdom.

This is interesting from a transhumanist perspective in that it suggests that the death of death would be the death of wisdom! I have seen some evidence for that in the incredible, shallow-minded selfishness of a certain subset of the transhumanist community -- people who are dead-set on having their own selves live forever, without any real thought as to why this might be valuable or what this might mean in a larger perspective. But of course, I don't really think death is the only or ultimate source of wisdom, though in a human context I can believe it's one of the main forces nudging us toward wisdom.

Paul Graham on Wisdom

One of the more interesting theories of wisdom I've run across (I found it a while ago for some random reason I've forgotten, and dug it up again last week) came from a contemporary blogger, Paul Graham:

http://paulgraham.com/wisdom.html

who distinguishes wisdom from intelligence in the following way:


"Wise" and "smart" are both ways of saying someone knows what to do. The difference is that "wise" means one has a high average outcome across all situations, and "smart" means one does spectacularly well in a few.

This explanation also suggests why wisdom is such an elusive concept: there's no such thing. "Wise" means something—that one is on average good at making the right choice. But giving the name "wisdom" to the supposed quality that enables one to do that doesn't mean such a thing exists. To the extent "wisdom" means anything, it refers to a grab-bag of qualities as various as self-discipline, experience, and empathy

Graham considers wisdom as partly a kind of de-biasing and cleansing of the mind, a notion that has some resonance with the modern notion of "Bayesian calibration" of the mind:

Recipes for wisdom, particularly ancient ones, tend to have a remedial character. To achieve wisdom one must cut away all the debris that fills one's head on emergence from childhood, leaving only the important stuff. Both self-control and experience have this effect: to eliminate the random biases that come from your own nature and from the circumstances of your upbringing respectively. That's not all wisdom is, but it's a large part of it. Much of what's in the sage's head is also in the head of every twelve year old. The difference is that in the head of the twelve year old it's mixed together with a lot of random junk.

Provocatively, Graham also posits that intelligence is quite different from wisdom, in that it has to do with accentuating rather than avoiding biases:

The path to intelligence seems to be through working on hard problems. You develop intelligence as you might develop muscles, through exercise. But there can't be too much compulsion here. No amount of discipline can replace genuine curiosity. So cultivating intelligence seems to be a matter of identifying some bias in one's character -— some tendency to be interested in certain types of things—- and nurturing it. Instead of obliterating your idiosyncrasies in an effort to make yourself a neutral vessel for the truth, you select one and try to grow it from a seedling into a tree.

To avoid confusion, from here on I'll sometimes refer to Graham's interpretation of these concepts as Graham-style wisdom and Graham-style intelligence, respectively.

There is an unclarity in Graham's essay as to the extent to which he thinks the kind of focusing and bias-accentuation that's part of Graham-style intelligence has to involve irrationality. My own view is that Graham-style intelligence definitely does NOT require an individual to be irrational, in the sense of making suboptimal judgments about a particular problem given the resources devoted to thinking about the problem. However, a finite system in a complex environment is always going to be irrational to some measure, due to not having enough resources to make a fully analysis of any complex situation. To the extent that Graham-style intelligence involves heavy focus on some particular set of topic areas, it's going to drain resources from other areas, thus making the mind less intelligent regarding these other areas.

So, in Graham's view, intelligence has to do with focusing loads of resources on processing in a handful of narrow domains that match one's innate biases, whereas wisdom has to do with evenly distributing processing across all the different domains in one's environment.

Along these lines Graham also notes (correctly, I think) that:

The wise are all much alike in their wisdom, but very smart people tend to be smart in distinctive ways.

As Graham conceives it, wisdom is basically equivalent to general intelligence: it's intelligence averaged across a variety of situations. In mathematics there exist various sorts of averages, some of which weight extreme values more heavily than others (these are p'th power averages). Graham's view would be that "wisdom" and "intelligence" are both estimates of general intelligence (defined as intelligence averaged over different domains/tasks), but with different sorts of averaging: in the case of intelligence, an averaging that pays especial attention to extremes (say a p-power average with p=5, or whatever); and in the case of wisdom, a more typical arithmetic averaging.

This is all sort of nice, but (as will become clear as the essay unfolds) I don't really think it gets at the crux of the matter.


Wisdom Goes Beyond the Individual

Another interesting perspective (that I also think doesn't get at the crux of the matter) is given in the paper "Meaning generation and artificial wisdom" with abstract

We propose an interpretation of wisdom in terms of meaning generation in social groups. Sapient agents are able to generate useful meanings for other agents beyond their own capability of generation of self-meanings. This makes sapient agents specially valuable entities in agent societies because they provide interagent reliable third-person meaning generation that provides some functional redundancy that contributes to enhance individual and social robustness and global performance.

Here wisdom is identified with the ability to generate meaning in the social group, going beyond meaning that is perceptible by the individual doing the meaning-generating. This harks back to Erikson's understanding of wisdom as related to identification with the world at large, beyond the mind/body.

This view also reminds me vaguely of Aldous Huxley's Perennial Philosophy, an attempt to distill the "wisdom teachings" of all the world's religions. In the Perennial Philosophy, wisdom teaches that the individual self is an illusion and all of us are one with the universe (and yet in a sense still distinct and individual.)

Mulling over all this, none of it really satisfied me. Of course, a folks concept like "wisdom" can't be expected to have a crisp and sensible formalistic definition ... but it still seemed to me that all the attempts at systematization and formalization I'd read about were missing some really essential aspects of the folk concept.

Wisdom, Cleverness and Skillfulness

And so, I came up with a totally different idea....

After a fair bit of musing, my mind kept drifting to the familiar distinction between declarative, procedural and episodic memory (drawn from textbook cognitive psych).

Remember:

• Declarative knowledge = knowledge of facts, conjectures, hypotheses (abstract or concrete)
• Procedural knowledge = knowledge of how to do things (could be physical, mental, social, etc.)
• Episodic knowledge = knowledge of stories that have occurred in the history of intelligent beings (oneself, others one knows, others one has heard about,...)

One interesting thought that popped into my head is: The concept of wisdom, in its folk-psychology sense, has a lot to do with the ability to solve problems that are heavily dependent on context, using intuition that's based on large-scale analysis of one's episodic-memory store.

Or, less geekily: Wisdom consists of making intelligent use of experience.

A subtlety here is that this need not be one's own experience. Direct experience may be the best way to acquire wisdom (and surely this is part of the reason that wisdom is commonly associated with age) but some rare folks are remarkably gifted at absorbing wisdom from the experience of others -- absorbed via observation, via reading, or conversation, or whatever.

More broadly, this train of thought leads me to a sort of fundamental trinity of aspects of intelligence: cleverness, skillfulness and wisdom.

There's cleverness, which is the ability to appropriately manipulate, create and absorb declarative knowledge toward one's goals. This declarative knowledge may be abstract, or it may be concrete facts. Declarative knowledge is largely symbolic in nature, and cleverness is largely founded on adeptness at symbol-manipulation.

There's skillfulness, which is the ability to effectively do stuff in service of one's goals. This covers physical skills but also highly abstract mental skills like writing an essay, proving a theorem, or closing a business deal.

In some domains skillfulness can exist in the total absence of cleverness. The vast majority of shred metal guitarists would seem to fit in this category (to choose a somewhat random example based on what's playing in my headphones at the moment). These guys are so damn skilled, yet there's not much adept manipulation of meaning in their solos, or compositions. Compare the typical shred guitarist to Yngwie Malmsteen or Buckethead, who are also massively skilled (and in similar ways) -- but who are also highly clever in their symbolic manipulation of the abstract patterns characterizing the concrete sonic forms they're so skilled at producing.

In other domains, it's really hard for cleverness and skillfulness to emerge in any way except exquisitely intercombined. Mathematics is an example. Procedural knowledge at doing proofs is needed for fully understanding complex proofs -- because so many steps are left out in proofs as typically written down, if you don't know how to do proofs, you won't be able to fill in all the gaps in your head when you read a proof, so you'll never get more than a general understanding. On the other hand, it's even more obvious that deep declarative understanding and manipulation-ability regarding mathematical content is necessary to do mathematical proofs. Math is a domain where procedural and declarative intelligence have got to work in extremely tight synergy.

Finally, there's wisdom, which as I'm conceiving it here is the ability to intelligently draw conclusions from a vast repository of data regarding specific situations.

Human minds tend to organize data regarding specific situations using story-like, "narrative" structure, so that in human practice, wisdom often takes the form of the ability to mine appropriate abstract patterns from a vast pool of remembered stories.

Of course, the operation of human episodic memory is largely constructive -- we don't actually grab experiential data out of some sort of neurological database; rather, we synthesize stories from fragmentary images, stories, and such. Wisdom is about synthesizing appropriate stories from large databases of partially-remembered, ambiguous, fractional stories -- and then, as appropriate, using these stories to guide the creation of declarative or procedural knowledge.

In mathematics, wisdom is closely related to what's called "mathematical maturity" ... the general sense of how mathematics is done. Mathematical maturity guides the mind to interesting problems and interesting concepts ... and helps you choose an overall proof strategy (whereas it's cleverness and skillfulness that help you carry out the proof).

The transition from {cleverness + skillfulness} to wisdom in music is epitomized to me by the mid-to-late John Coltrane ... the Coltrane of "My Favorite Things" and "A Love Supreme." These are the solos of a man who has listened so much and played so much that he's disassembled thousands of different musical narratives and reassembled them to tell different kinds of stories, like no one ever told before. So much richer than the merely clever, skillful and emotionally moving solos of the early Coltrane. Certain works of great art manage to be intensely personal and dramatically universal at the same time, and
this often results from wisdom in the sense I'm defining it here.

Note that a mature mathematician or a world-changing jazz soloist need not be "wise" in the sense of a Taoist sage. The classical conception of wisdom has to do with making intelligent judgments based on large stores of experience in everyday human life. In the old days this was pretty much the only experience there was -- everyday human life plus various shamanic and psychedelic experiences.... But now the human world has become far more specialized, and it's possible to have a specialized wisdom, because it's possible to have a huge and rich store of episodic knowledge that's restricted to some special domain, like music or mathematics, or even a sufficiently complex game like Go or chess.

This vision of wisdom would seem to contradict Graham's, cited above -- he views wisdom as related to the ability to achieve goals over a broad variety of domains, in contract to intelligence which he conceives as a more narrowly domain-specialized intelligence.

But I don't think the contradiction is total.

I think that within a sufficiently rich and complex domain, one requires wisdom as I've defined it in order to achieve a really high level of intelligence. Learning skills and manipulating symbols is not enough. Direct and intelligent mining of massive experience-stores is needed.

I also think that wisdom, even if achieved initially and primarily within a certain domain, has a striking power to transcend domains. There are a lot of universal patterns among large stores of stories, no matter what the domain.

But even if the wisdom achieved by a great mathematician or chess player or jazz soloist helps that person to intuitively understand the way things work in other domains, this won't necessarily lead them to practical greatness in these other domains -- great achievement seems to require a synthesis of wisdom with either cleverness or skillfulness, and in some domains (like math or jazz improvisation) all three.

Defined-Problem versus Contextual Intelligence

Next, what does all this have to do with artificial intelligence?

One of the lessons learned in the last few decades of AI practice is that there is a pretty big difference between:

1. Defined-problem intelligence: Problem-solving that occurs "after a crisply-defined problem statement has been identified", versus
2. Contextual intelligence: problem-solving that is mainly concerned with interpreting general goals in the context of a complex situation, and, "figuring out what the context-specific problem is, in the first place" -- i.e. figuring out what crisply-defined problem, if solved in the relevant context, is likely to work toward the general goals at hand

I think this might be a more useful and more precise distinction than the "narrow AI" versus "general AI" distinction that I've often made before. It's ultimately getting at the same thing, but it's putting the point in a better way, I think.

What's narrow about "narrow AI" systems like chess-playing programs and medical diagnostic expert systems isn't merely that they're focused on specific, narrow domains. It's the fact that they operate based on defined-problem intelligence. It happens, though, that in some sufficiently specialized domains, defined-problem intelligence is enough to yield ass-kicking performance. In other domains it's not -- because in these other domains, figuring out what the problem is, is basically the problem.

I suggest that defined-problem intelligence is focused on declarative and procedural knowledge: i.e. it consists of cleverness or skillfulness or some combination thereof.

Logical reasoning systems, for example, are focused on declarative knowledge, and possess in some cases great facility at manipulating declarative knowledge.

Evolutio