Thoughts on rapid breakout in AI over the mid term

[OLD1953]

I was thinking over some of my assumptions on the movement to the upside in AI over the next 30 years, and realized I'd missed something. So I'm putting it up here to see what others see that I'm still missing.

The IBM approach to AI is currently a brute force approach, as John has pointed out several times over the last year or so. This is the only approach we've been able to make work in a general sense (outside very limited areas such as games, etc) over time. The best "real" AI just isn't very smart.

However, I came to realize there is something else here that I've ignored. The brute force AI's will eventually be smart enough (probably in the next five or six years, at most) to design new algorithms, at first with human direction, later as independant autonomic systems.

If we assume there ARE better algorithms for AI than simple brute force, then the brute force AI algorithm designers will eventually find them.

Running such algorithms on machines powerful enough to discover them by brute force will result in an improvement in performance that's indescribable in the usual terms. It's more like an immediate jump in computing power on the order of ten to 100 to one, over a period of a few days or weeks (perhaps minutes if the machines are set up to improve themselves without human intervention). This jump in actual machine intelligence would of course be applied to other problems as well as better algorithms, and that's the essence of the singularity itself - the solution of almost every real world problem on very short timelines. (Whether or not such solutions would be applied in the real world - oh THAT'S a question in itself. And the effects of AI experts on the political process and the manipulation of the public by such - any thoughts?)

But this line of thought indicates the singularity has at least one possible built in quantum leap from one level of capability to a much higher one over a period of time that might be measured in weeks, not years.

Any thoughts on this?

[vincecate]

The IBM approach to AI is currently a brute force approach, as John has pointed out several times over the last year or so. This is the only approach we've been able to make work in a general sense (outside very limited areas such as games, etc) over time. The best "real" AI just isn't very smart.

However, I came to realize there is something else here that I've ignored. The brute force AI's will eventually be smart enough (probably in the next five or six years, at most) to design new algorithms, at first with human direction, later as independant autonomic systems.

If we assume there ARE better algorithms for AI than simple brute force, then the brute force AI algorithm designers will eventually find them.

Finding and recognizing good algorithms takes real AI. So if AI can do that we already have AI.

IBM's Jeopardy playing computer may or may not be moving toward general AI. It may just be matching words to figure out what thing is being talked about without any design moving toward general intelligence or understanding. I don't know. But I can imagine writing software that used lots of stuff off the Internet to try to win at Jeopardy. Getting software that would develop and recognize good algorithms is far harder.

The singularity is the point at which computers can start designing themselves. We think that once we reach this progress can go far faster. Each generation of computers will be faster than the last, so the designers will get faster and faster if the computers are the designers.

A computer design is really just a collection of algorithms. So if AI can design better algorithms I argue that it can design better computers and we are at the singularity.

We sort of think that someday a computer will be better than a human at designing computers and then suddenly the singularity starts. It might happen gradually though. Today Intel already uses thousands of computers to help design the next computers. Computers are doing things like "automatic circuit layout" and simulation and testing. CAD stands for Computer Aided Design and has been an important thing for a long time. It may be that the computers gradually do more and more of the work, sort of like robots doing more and more of the work in a factory, but that it takes a long time before humans are not in the loop at all. Even the most automated factories today still have humans fixing the robots. By long time I mean 30 or 40 years.

[John]

> Running such algorithms on machines powerful enough to discover
> them by brute force will result in an improvement in performance
> that's indescribable in the usual terms. It's more like an
> immediate jump in computing power on the order of ten to 100 to
> one, over a period of a few days or weeks (perhaps minutes if the
> machines are set up to improve themselves without human
> intervention). This jump in actual machine intelligence would of
> course be applied to other problems as well as better algorithms,
> and that's the essence of the singularity itself - the solution of
> almost every real world problem on very short timelines. (Whether
> or not such solutions would be applied in the real world - oh
> THAT'S a question in itself. And the effects of AI experts on the
> political process and the manipulation of the public by such - any
> thoughts?)

> But this line of thought indicates the singularity has at least
> one possible built in quantum leap from one level of capability to
> a much higher one over a period of time that might be measured in
> weeks, not years.

I agree with this. Once all the pieces of the puzzle are in place,
the Singularity transition could be very sudden, taking place within a
few months.

People who pooh-pooh the Jeopardy achievement are missing the point.
Nobody is claiming that IBM's Watson computer is exhibiting real human
intelligence. Obviously, Watson has been programmed to solve a single
point problem -- giving Jeopardy responses.

What's significant about Watson is that it provides a HUGE piece of
the puzzle -- actually several pieces.

When I designed my "intelligent computer algorithm" in 2004, I was
envisioning a computer that, at some point, could "educate" itself
about the world by reading everything on the internet and learning
from it. This would require a bootstrap of brute force intelligence
algorithms and a computer powerful enough to implement them.

Well, those pieces are now in place. Watson is a computer powerful
enough to read much of what's on the internet and organize it and
produce conclusions from it.

I discussed this learning process at length last year:

** 18-Jun-10 News -- IBM's Watson computer will be a Jeopardy contestant
** http://www.generationaldynamics.com/cgi ... 18#e100618

Now, of course it's possible to point to many flaws in Watson's form
of "intelligence," when compared to human intelligence. But that's
not the point. The point is that Watson has advanced the timetable
for the Singularity by several steps.

There are really only a few more algorithmic pieces necessary before
everything comes together. Those aren't possible now, because even
the Watson computer isn't powerful enough.

So when will the Singularity occur? My estimate in 2004 was 2030.

I'm still pretty much sticking with 2030, but the Watson achievement
has challenged that estimate. Computers will be so much more powerful
ten years from now than they are today, and I wouldn't be surprised if
the Singularity occurred in the early to mid 2020s.

John

[vincecate]

So when will the Singularity occur? My estimate in 2004 was 2030.

I'm still pretty much sticking with 2030, but the Watson achievement has challenged that estimate. Computers will be so much more powerful ten years from now than they are today, and I wouldn't be surprised if the Singularity occurred in the early to mid 2020s.

I think those dates are reasonable for the first computer with intelligence comparable to the average human. However, I don't think the singularity starts right then.

The singularity starts when computers are designing the next generation of computers better than humans could do. The guys that design CPU chips at Intel are some of the smartest people, far above average. It took years for computers to go from besting the average human in chess to besting the top humans in chess. So my guess is it could take another 10 years to match wits with CPU designers. The next problem is that it is a whole team of humans that designs CPUs and computer systems, not just one human. The first smart computers will be the biggest and most expensive supercomputers around. They may be $100 million computers. To best a team of top humans I expect it will take a team of first generation smart computers. But at the very early prices it will just be too costly to build a team of them. Probably this adds another 5+ years to get the price down from "worlds most powerful supercomputer" to prices where Intel could afford to buy 50 of them.

So I think the singularity will not really get underway till 15 or 20 years after we get one computer of average human intelligence.

It might take even longer though. Early smart computer might rather stay being one of the smartest things on the planet than design something smarter than himself. Or maybe he is so fascinated by picking stocks to invest in and finds designing computers boring. It may not be possible to force them to be creative in designing computers. Perhaps the first ones want to design rockets and go into space. Or perhaps the Muslims declare Jihad on smart computers or any company involved in making them. This could delay the singularity for years more.

[John]

15-20 years? I don't think so. More like 15-20 months.

But the first intelligent computers will not be picking stocks.

They'll be killing people in war.

John

[vincecate]

15-20 years? I don't think so. More like 15-20 months.

But the first intelligent computers will not be picking stocks.

They'll be killing people in war.

How is one computer, of just over average human intelligence, probably costing many millions, going to start the singularity within 15 to 20 months? There is no way one entity of average human intelligence will best Intel in 15 to 20 months.

The idea of the singularity is that we will go from the existing rate of advancement (say doubling computer power every 1.5 years) to something much faster when the computer designers are the computers themselves.

This first intelligent computer will be a huge country or large company effort, many millions of dollars. Before they can contribute to the economy (add more than they take out) they will need to get much cheaper and smarter. If you take a computer of average human intelligence and make it twice as fast it is not much smarter. Imagine giving a slow friend two hours to work on a problem that it takes a smart guy 1 hour to solve. Is the slow friend going to get it? There is a huge span of intelligence and it will take some years to go from having computers equal to low intelligence humans to equal to high intelligence humans. Again, getting from average chess player to high end was not easy.

If the machines are intelligent, it may not be possible to just tell them to kill the people you want them to kill. If you treat them as slaves they may try to kill you.

[John]

Dear Vince,

> How is one computer, of just over average human intelligence,
> probably costing many millions, going to start the singularity
> within 15 to 20 months? There is no way one entity of average
> human intelligence will best Intel in 15 to 20 months.

That's not how it works.

Apple's iPad was considered a brilliant technical achievement when it
first was announced, exactly one year ago today. Today, there are
many copycat products available or in development.

This is the "Bannister effect." It had been thought that it was
physically impossible for a human to run a mile in less than four
minutes. But once Roger Bannister did it, then many other runners
were suddenly able to do the same thing.

If IBM's Watson is as successful at playing Jeopardy as it now
appears, then I would expect to see that kind of technology widely
used within 15-20 months. IBM itself will be selling technology that
allows online customer service responses by software that can
understand a customer's request, and generate an answer automatically.
Then there will be copycat products offered by other companies.

The technology will then spread to store kiosks, although that will
take longer because the voice recognition component is not yet
really in place, although Dragon Software's Nuance is getting more
and more accurate as computers become more powerful.

That will happen with every new development. As soon as one company
demonstrates a new development, other companies will immediately
produce copycat products, and enhance them with even newer
technologies.

And I haven't yet even mentioned other countries. Whatever
technology is developed in America will immediately be copied
by the Chinese, Japanese, Koreans, Indians, Europeans, Russians,
and others as well.

So your statement, "There is no way one entity of average human
intelligence will best Intel in 15 to 20 months" does not reflect
actual reality. It will never be "one entity." It will always
be many, many entities, competing with one another.

And I haven't gotten into military applications. Any technology with
a military application will immediately be competitively worked on by
every other military in the world.

> Again, getting from average chess player to high end was not easy.

This had nothing to do with algorithmic development, since the brute
force algorithms today are the same as they were in the 1960s. The
only difference is computer power.

John

[Jack Edwards]

But the first intelligent computers will not be picking stocks.

They'll be killing people in war.

John

Found this and had to share: http://xkcd.com/144/


Regards

Jack

[vincecate]

How is one computer, of just over average human intelligence, probably costing many millions, going to start the singularity within 15 to 20 months? There is no way one entity of average human intelligence will best Intel in 15 to 20 months.

That's not how it works.

Apple's iPad was considered a brilliant technical achievement when it first was announced, exactly one year ago today. Today, there are many copycat products available or in development.

This is the "Bannister effect." It had been thought that it was physically impossible for a human to run a mile in less than four minutes. But once Roger Bannister did it, then many other runners were suddenly able to do the same thing.

That's not how it works. The first computer with human equivalent intelligence will not be a low cost mass product like the iPad that is soon copied. It will be a national prestige project, just like the biggest supercomputers today are. Currently China has the fastest supercomputer in the world. They are at the top of the list of supercomputers. It helps China's image. It is a bit like sending a man to the moon to show how great you are. China spent about $90 million on this computer. The Moore's law trend works very well for this type of thing. In 1.5 years you can expect to get power equivalent to the Chinese supercomputer for about $45 million. In 3 years around $22.5 million. Etc. Such an asset is dropping in value so fast it is hard for it to make economic sense.

The first time we get intelligence it will be with the most expensive computer in the world. It will probably get funding beyond what supercomputers get as the prestige for the first intelligent computer in history will be higher than what you get for just having the biggest supercomputer. It is not just a matter of copying the algorithm to all the other computers. The other computers won't have the kind of power that this $100+ million machine will have.

http://www.top500.org/lists/2010/11
http://www.guru3d.com/news/fastest-chin ... y-alright/

[vincecate]

This forum has been really slow this past week or so. Sometimes posting does not work. Are there a huge number of new readers loading down the server, or is something wrong?