308. AI: The Beast or Jerusalem? | Jonathan Pageau & Jim Keller

[0] Hello everyone watching on YouTube or listening on Associated Platforms.

[1] I'm very excited today to be bringing you two of the people I admire most intellectually, I would say, and morally for that matter, Jonathan Pajot and Jim Keller, very different thinkers.

[2] Jonathan Paggio is a French -Canadian maturgical artist and icon carver known for his work, featured in museums across the world.

[3] He carves Eastern Orthodox, among other traditional images, and teaches an online carving class.

[4] He also runs a YouTube channel, This Symbolic World, dedicated to the exploration of symbolism across history and religion.

[5] Jonathan is one of the deepest religious thinkers I've ever met.

[6] Jim Keller is a microprocessor engineer known very well.

[7] in the relevant communities and beyond them for his work at Apple and AMD, among other corporations.

[8] He served in the role of architect for numerous game -changing processors, has co -authored multiple instruction sets for highly complicated designs and is credited for being the key player behind AMD's renewed ability to compete with Intel in the high -end CPU market.

[9] In 2016, Keller joined Tesla becoming vice president of autopilot hardware engineering.

[10] In 2018, he became a senior vice president for Intel.

[11] In 2020, he resigned due to disagreements over outsourcing production, but quickly found a new position at Tens Torrent as chief technical officer.

[12] We're going to sit today and discuss the perils and promise of artificial intelligence.

[13] And it's a conversation I'm very much looking forward to.

[14] So welcome to all of you watching.

[15] listening.

[16] I thought it would be interesting to have a three -way conversation.

[17] Jonathan and I have been talking a lot lately, especially with John Verveci and some other people as well, about the fact that it seems necessary for us to view, for human beings to view the world through a story.

[18] In fact, that when we describe the structure that governs our action and our perception, that is a story.

[19] And so we've been trying to puzzle out, I would say, to some degree, on the religious front, what might be the deepest stories?

[20] And I'm very curious about the fact that we perceive the world through a story, human beings do.

[21] And that seems to be a fundamental part of our cognitive architecture and of cognitive architecture in general, according to some of the world's top neuroscientists.

[22] And I'm curious, and I know Jim is interested in cognitive processing and in building systems that, in some sense seem to run in a manner analogous to the manner in which our brains run.

[23] And so I'm curious about the overlap between the notion that we have to view the world through a story and what's happening on the AI front.

[24] There's all sorts of other places that we can take the conversation.

[25] So maybe I'll start with you, Jim.

[26] Do you want to tell people what you've been working on and maybe give a bit of a background to everyone about how you conceptualize artificial intelligence?

[27] Yeah, sure.

[28] So first, I'll say technically I'm not an artificial intelligent researcher.

[29] I'm a computer architect, and I'd say my skill set goes from, you know, somewhere around the atom up to the program.

[30] So we make transistors out of atoms, we make logical gates out of transistors, we make computers out of logical gates.

[31] We run programs on those.

[32] and recently we've been able to run programs fast enough to do something called an artificial intelligence model or a neural network, depending on the way to say it.

[33] And then we're building chips now that run artificial intelligence models fast, and we have a novel way to do it, the company I work at.

[34] But lots of people are working on it, and I think we were sort of taken by surprise what's happened in the last five years, how quickly models started to do interesting and intelligent seeming things.

[35] There's been an estimate that human brains do about 10 to the 18th operations a second.

[36] It sounds like a lot.

[37] It's a billion billion operations a second.

[38] And a little computer, you know, the processor in your phone probably does 10 billion operations a second.

[39] second, you know, ish.

[40] And then if you use the GPU, maybe 100 billion, something like that.

[41] And big modern AI computers, like Open AI users like OpenAI uses or Google or somebody, they're doing like 10 to the 16th, maybe slightly more operations a second.

[42] So they're within a factor of a hundred of a human brain's raw computational ability.

[43] And by the way, that could be completely wrong, our understanding of how the human brain does computation.

[44] could be wrong, but lots of people have estimated based on number of neurons, number of connections, how fast neurons fire, how many operations a neuron firing seems to involve.

[45] I mean, the estimates range by a couple words of magnitude, but when our computers got fast enough, we started to build things called language models and image models that do fairly remarkable things.

[46] So what have you seen in the last few years that's been indicative of this, of the change that you describe as revolutionary?

[47] What are the, what are computers doing now that you, that you found surprising because of this increase in speed?

[48] Yeah, you can have a language model read a 200 ,000 word book and summarize it fairly accurately.

[49] So it can extract out the gist?

[50] The gist of it.

[51] Can it do that with fiction?

[52] Yeah.

[53] Yeah, and I'm going to introduce you to a friend who took a language model and changed it and fine -tuned it with Shakespeare and used at the right screenplays that are pretty good.

[54] And these kinds of things are really interesting.

[55] And then we were talking about this a little bit earlier.

[56] So when computers do computations, you know, a program will say add A equal B plus C. The computer does those operations on representations of information, ones and zeros.

[57] It doesn't understand them at all.

[58] The computer has no understanding of it.

[59] But what we call a language model translates information like words and images and ideas into a space where the program, the ideas, and the operation that does on them, are all essentially the same thing.

[60] Right.

[61] So a language model can produce words and then use those words as inputs.

[62] And it seems to have an understanding of what those words are, which is very different from how a computer.

[63] I'm curious about the language models.

[64] I mean, my sense of, at least in part, how we understand a story, is that maybe we're watching a movie, let's say, and we get some sense of the character's goals, and then we see the manner in which that character perceives the world, and we, in some sense, adopt his goals, which is to identify with the character.

[65] And then we play out a panoply of emotions and motivations on our body because we now inhabit that goal space and we understand the character as a consequence of mimicking the character with our own physiology.

[66] And you have computers that can summarize the gist of a story but they don't have that underlying physiology.

[67] Well, first of all, it's a theory that your physiology has anything to do with it.

[68] You could understand the character's goals and then get involved in the details of the story.

[69] And then you're predicting the path of the story and also having expectations and hopes for the story.

[70] Yeah.

[71] And a good story kind of takes you on a ride because it teases you with doing some of the things you expect, but also doing things that are unexpected.

[72] And possibly that creates emotional.

[73] Yeah, it does.

[74] It does.

[75] So in an AI model, so you can easily have a set of goals.

[76] So you have your personal goals.

[77] And then when you watch the story, you have those goals.

[78] You put those together.

[79] Like, how many goals is that?

[80] Like, the story's goals and your goals, hundreds, thousands?

[81] Those are small numbers, right?

[82] Then you have the story.

[83] The AAL model can predict the story, too, just as well as you can.

[84] How do you...

[85] And...

[86] That's the thing that I find mysterious is that it...

[87] As the story progresses, it can look at the error between what it predicted and what actually happened.

[88] And then iterate on that.

[89] Right.

[90] So you would call that emotion.

[91] emotional excitement, disappointment.

[92] Anxiety.

[93] Yeah, definitely.

[94] Well, a big part of what anxiety does seem to be is discrepancy.

[95] Like some of those states are manifesting your body because you trigger hormone cascades and a bunch of stuff.

[96] But you also can just scan your brain and see that stuff move around.

[97] Right.

[98] Right.

[99] And, you know, the AI model can have an error function and look at the difference between what it expected it and not.

[100] And you could call that the emotional state.

[101] Yeah.

[102] Well, I just talked with the direct.

[103] speculation.

[104] No, no, I think that's accurate.

[105] But, you know, we can make an AI model that could predict the result of a story probably better than the average person.

[106] Some people are really good at, you know, they're rarely well -educated about stories or they know the genre or something, but, you know, these things, and what they see today is the capacity of the models is, if you say start describing it a lot, it will make sense for a while, but it will slowly stop making sense.

[107] but that that's possible that's simply the capacity of the model right now and the model's not well grounded enough in a set of the same goals in reality or something to make sense for a lot so what do you think would happen Jonathan this is this is I think associated with the kind of things that we've talked through to some degree so one of my hypotheses let's say about deep stories is that they're, they're meta -jists in some sense.

[108] So you can imagine 100 people telling you a tragic story, and then you could reduce each of those tragic stories to the gist of the tragic story, and then you could aggregate the gists, and then you'd have something like a meta -tragedy.

[109] And I would say, the deeper, the gist, the more religious -like the story gets.

[110] And that's part of, it's that idea as part of the reason that I wanted to bring you guys together.

[111] I mean, one of the things that what you just said makes me wonder is, imagine that you took Shakespeare and you took Dante and you took, like, the canonical Western writers, and you trained an AI system to understand the structure of each of them.

[112] And then, and now you have, you could pull out the summaries of those structures, the gists, and then couldn't you pull out another gist out of that?

[113] So it would be like the essential element of Dante and Shakespeare.

[114] And I want to hear what Jonathan's like so far.

[115] So here's one funny thing to think about.

[116] You use the word pull out.

[117] So when you train a model to know something, you can't just look in it and say, what is it?

[118] No. You have to query it.

[119] Right.

[120] You have to ask.

[121] Right.

[122] Right.

[123] What's the next sentence in this paragraph?

[124] What's the answer to this question?

[125] There's the thing on the Internet.

[126] now called prompt engineering.

[127] And it's the same way, I can't look in your brain to see what you think.

[128] Yeah.

[129] I have to ask you what you think.

[130] Because if I killed you and scanned your brain and got the current state of all the synapses and stuff, A, you'd be dead, which would be sad.

[131] And B, I wouldn't know anything about your thoughts.

[132] Your thoughts are embedded in this model that your brain carries around.

[133] And you can express it in a lot of ways.

[134] And so you could add.

[135] How do you train – this is my big question is – I mean, because the way that I've been seeing it until now is that artificial intelligence is – it's based on us.

[136] It's not – it doesn't exist independently from humans, and it doesn't have care.

[137] The question would be, why does the computer care?

[138] Yeah, that's not true.

[139] Why does the computer care to get the gist of the story?

[140] Well, yeah, so I think you're asking kind of the wrong question.

[141] So you can train an AI model.

[142] on like the physics and reality and images in the world, just with images.

[143] And there are people who are figuring out how to train a model with just images, but the model itself still conceptualizes things like tree and dog and action and run, because those all exist in the world.

[144] Right, so, and you can actually train, so, and when you train a model with all the language and words, so all information has structure and I know you're a structure guy from your video so if you look around you at any image every single point you see makes sense yeah right it's a teleological structure it's like a it's a purpose laid in structure right so this is something so it turns out all the words that have ever been spoken by human beings also have structure right right and and so physics has structure, and it turns out that some of the deep structure of images and actions and words and sentences are related.

[145] Like, there's actually a common core of, like, imagine there's like a knowledge space.

[146] And sure, there's details of humanity where, you know, they prefer this, this accent versus that.

[147] Those are kind of details, but they're coherent in the language model.

[148] But the language models themselves are coherent with our words.

[149] ideas and humans are trained in the world just the way the AI models are trained in the world like a little baby as it's learning looking around it's training on everything it sees when it's very young and then its training rate goes down and it starts interacting what it's with what it's learning and interacting with the people but it's trying to survive it's it's it's trying to live it has like it has a the infant or the child has neurons aren't trying the the weights and the neurons aren't trying to live what you're trying to do is reduce the error.

[150] So neural networks generally are predictive things, like what's coming next, what makes sense, you know, how does this work?

[151] And when you train an AI model, you're training it to reduce the error in the model.

[152] And if your model's big...

[153] Okay, let me ask you about that.

[154] So, well, first of all...

[155] And so babies are doing the same thing.

[156] like they're looking at stuff go around and in the beginning their neurons are just randomly firing but as it starts to get object permanence and look at stuff it starts predicting what will make sense for that thing to do and when it doesn't make sense it'll update its mall basically it compares its prediction to the events and then it will adjust its prediction so in a story prediction model the AI would predict the story then compare it to its prediction and then fine tune itself slowly as it trains its itself.

[157] Or in reverse, you could ask it to say, given the set of things, tell the rest of the story, and it could do that.

[158] And that's what, and the state of it right now is there are people having conversations with this that are pretty good.

[159] So I talked to Carl Fristin about this prediction idea in some detail.

[160] And so Fristin, for those of you who are watching and listening, is one of the world's talk neuroscientists.

[161] And he's developed an entropy enclosure model of conceptualization, which is analogous to one that I was working on, I suppose, across approximately the same time frame.

[162] So the first issue, and this has been well established in the neuropsychological literature for quite a long time, is that anxiety is an indicator of discrepancy between prediction and actuality.

[163] And then positive emotion also looks like a discrepancy reduction indicator.

[164] So imagine that you're moving towards a goal.

[165] and then you evaluate what happens as you move towards the goal.

[166] And if you're moving in the right direction, what happens is what you might say, what you expect to happen.

[167] And that produces positive emotion.

[168] And it's actually an indicator of reduction in entropy.

[169] That's one way of looking at it.

[170] Then the point is that the - Yeah, you have a bunch of words in there that are psychological definitions of states.

[171] But you could say there's a prediction and error of prediction.

[172] Yes.

[173] And you're reducing error.

[174] Yes.

[175] But what I'm trying to make a case for is that your emotions directly map that, both positive and negative emotion, look like there's signifiers of discrepancy reduction, both on the positive and negative emotion side.

[176] But then there's a complexity that I think is germane to part of Jonathan's query, which is that...

[177] So the neuropsychologists and the cognitive scientists have talked a lot of long time about expectation, prediction, and discrepancy reduction.

[178] but one of the things they haven't talked about is it isn't exactly that you expect things it's that you desire them you want them to happen because you could imagine that there's in some sense a literally infinite number of things you could expect and we don't strive only to match prediction we strive to bring about what it is that we want and so we have these preset systems that are teleological that are motivational systems well I mean it depends Like, if you're sitting idly on the beach, like in a bird flies by, you expect it to fly along in a regular path.

[179] Right.

[180] You don't really want that to happen.

[181] Yeah, but you don't want it to turn into something that could peck out your eyes either.

[182] Sure.

[183] So that's a want.

[184] Yeah.

[185] But you're kind of following it with your expectation to look for a discrepancy.

[186] Yes.

[187] Now, you'll also have a, you know, depends on the person, somewhere between 10 and a million desires, right?

[188] and then you also have fears and avoidance.

[189] And those are context.

[190] So if you're sitting on the beach with some anxiety that the birds are going to swerve at you and take your eyes out.

[191] So then you might be watching it much more attentively than somebody who doesn't have that worry, for example.

[192] But both of you can predict where it's going to fly and you will both notice a discrepancy.

[193] The motivations, one way of conceptualizing fundamental motivation is they're like a prior prediction, domains, right?

[194] And so it helps us narrow our attentional focus because I know, I know when you're sitting and, and you're not motivated in any sense, you can be doing just in some sense trivial expectation computations, but often we're in a highly motivated state.

[195] And what we're expecting is bounded by what we desire and what we desire is oriented, as Jonathan pointed out, towards the fact that we want to exist.

[196] And one of the things I don't understand and wanted to talk about today is how the computer models, the AI models, can generate intelligible sense without this, without mimicking that sense of motivation.

[197] Because you've said, for example, they can just derive the patterns from observations of the objective world, but there's a...

[198] So, let's, so, again, I don't want to do all the talking, but so AI generally speaking, like when I first learned it about it, had two, two behaviors.

[199] So they call it inference in training.

[200] So inferences, you have a trained model.

[201] So you say, you give it a picture and say, is there a cat in it?

[202] And it tells you where the cat is.

[203] That's inference.

[204] The model has been trained to know where a cat is.

[205] And training is the process of giving the input and an expected output.

[206] And when you first start training the model, gives you garbage out, like an untrained brain.

[207] And then you take the difference between the garbage output and the expected output and call that the error.

[208] And then they invent the big revelation was something called back propagation with gradient descent.

[209] But that means take the error and divide it up across the layers and correct those calculations so that when you put a new thing in, it gives you a better answer.

[210] And then to somewhat my astonishment, if you have a model of sufficient capacity and you train it with 100 million images, if you give it a novel image and say, tell me where the cat is, it can do it.

[211] That's called, that's called, so training is the process of doing a pass with an expected output and propagating an error back through the network and inferences the behavior of putting this something in and getting an output.

[212] I think, am I, like, I'm really pulling.

[213] But there's a third piece, which is what the new models do, which is called generative, it's called a generative model.

[214] So, for example, say you put it in a sentence and you say, predict the next word.

[215] This is the simplest thing.

[216] So it predicts the next word.

[217] So you add that word to the input and now say, predict the next word.

[218] So it contains the original sentence and the word you generated.

[219] And it keeps generating words that make sense in the context of the original word in addition.

[220] Right, right, right.

[221] This is the simplest basis.

[222] And then it turns out you can train this to do lots of things.

[223] You can change it to summarize a sentence.

[224] You can train it to answer a question.

[225] There's a big thing about, you know, like Google every day has hundreds of millions of people asking it questions and giving answers and then rating the results.

[226] You can train a model with that information.

[227] So you can ask it a question and it gives you a sensible answer.

[228] But I think in what you said, I actually have the issue that has been going through my mind so much is when you said, you know, people put it.

[229] the question, and then they rate the answer.

[230] My intuition is that the intelligence still comes from humans in the sense that it seems like in order to train whatever AI, you have to be able to give it a lot of power and then say at the beginning, this is good, this is bad, this is good, this is bad, like reject certain things, accept certain things, in order to then reach a point when then you train the AI.

[231] And so that's what I mean about the care.

[232] So the care will come from humans because the care is the one giving it the value, saying this is what is valuable, this is what is not valuable in your calculation.

[233] So when they first, so there's a program called AlphaGo that learned how to play Go better than a human.

[234] So there's two ways to train the model.

[235] One is they have a huge database of lots of go games with good winning moves.

[236] So they trained the model with that.

[237] And that worked pretty good.

[238] And they also took two simulations of go, and they did random moves.

[239] And all that happened was these two simulators played one go game, and they just recorded whichever moves happened to win.

[240] And it started out really horrible.

[241] And they just started training the model, and this is called adversarial learning.

[242] This is not a particular adversarial.

[243] It's like, you know, you make your moves randomly, and you train a model.

[244] And so they train multiple models.

[245] and over time those models got very good and they actually got better than human players because the humans have limitations about what they know whereas the models could experiment in a really random space and go very far.

[246] Yeah, but experiment towards the purpose of winning the game.

[247] Yes, well, but you can experiment towards all kinds of things, it turns out.

[248] And humans are also trained that way.

[249] Like when you were learning, you were reading, you were said, this is a good book, this is a bad book, This is good sentence construction.

[250] It's good spelling.

[251] So you've gotten so many error signals over your life.

[252] Well, that's what culture does in large parts.

[253] And culture does that.

[254] Religion does that.

[255] Your everyday experience does that.

[256] Your family.

[257] So we embody that.

[258] Yeah.

[259] And we're all, and everything that happens to us, we process it on the inference pass, which generates outputs.

[260] And then sometimes we look at that and say, that's unexpected or that got a bad result or that got bad feedback and then we back propagate that and update our models so could could there be really well -trained models can then train other models so the humans right now are the smartest people in the world so the biggest question the biggest question that i that that comes now based on what you said is because my my main point is to try to show how it seems like artificial intelligence is always an extension of human intelligence.

[261] Like, it remains an extension of human intelligence.

[262] And maybe the way to...

[263] That won't be true at all.

[264] So do you think that at some point, the artificial intelligence will be able to...

[265] Because the goals, recognizing cats, you know, writing plays, all these goals are goals which are based on embodied human existence.

[266] Could you train...

[267] Could an AI at some point develop a goal which would be uncomprehensible to humans because of its own existence.

[268] Yeah, for example, there's a small population of humans that enjoy math, right?

[269] And they are pursuing, you know, adventures in math space that are incomprehensible to 99 .99 % of humans.

[270] But they're interested in it, and you could imagine, like, an AI program, working with those mathematicians and coming up with very novel math ideas and then interacting with them.

[271] But they could also, you know, if some AIs were elaborating out really interesting and detailed stories, they could come up with stories that are really interesting.

[272] We're going to see it pretty soon, like all of art, moviemaking, and everything.

[273] A story that is interesting only to the AI and not interesting to us.

[274] That's possible.

[275] So stories are like, I think, some high -level information space.

[276] So the computer, age of big data, there's all this data running on computers, where only humans understood it, right?

[277] Computers don't.

[278] So AI programs are now at the state where the information, the processing, and the feedback loops are all kind of in the same space.

[279] They're still, you know, relatively rudimentary to humans.

[280] I guess some AI programs in certain things are better than humans already, but for the most part, they're not.

[281] But it's moving really fast.

[282] So you could imagine, you I think in five or ten years, most people's best friends will be AIs.

[283] And, you know, they'll know you really well and they'll be interested in you.

[284] And, you know, it's...

[285] Unlike your real friends.

[286] Real friends are problematic.

[287] They're only interested in you when you're interested in you.

[288] Yeah, yeah, real friends are...

[289] The AI systems will love you even when you're dull and miserable.

[290] Well, there's, and there's so much idea space to explore.

[291] And humans have a wide range.

[292] Some humans like to go through their everyday life doing their everyday things.

[293] And some people spend a lot of time, like you, a lot of time reading and thinking and talking and arguing and debating.

[294] You know, and, you know, there's going to be, like, say, a diversity of possibilities with what a thinking thing can do when the thinking is fairly unlimited.

[295] So I'm curious about, I'm still, I'm curious in pursuing this issue that Jonathan has been developing.

[296] So there's a literally infinite number of ways, virtually infinite number of ways, that we could take images of this room.

[297] Right now, if a human being is taking images of this room, they're going to be, they're going to sample a very small space of that infinite range of possibilities.

[298] Because if I was taking pictures in this room, in all likelihood, I would take pictures of objects that are identifiable to human beings, that are functional to human beings, at a level of focus that makes those objects clear.

[299] And so then you could imagine that the set of all images on the internet has that implicit structure of perception built into it.

[300] And that's a function of what human beings find useful.

[301] I mean, I could take a photo of you that the focal depth was here and here and here and here and two inches past you.

[302] Now, I suppose you could...

[303] There's a technology for that called light fields.

[304] Okay.

[305] So then you could, if you had that picture properly done, And then you could move around at an image and see.

[306] But yeah, fair enough.

[307] I get your point.

[308] Like the human recorded data has...

[309] Has our biology built into it?

[310] Has our biology built into it, but also unbelievably detailed encoding of how physical reality works.

[311] Right.

[312] So every single pixel in those pictures, even though you kind of selected the view, the focus, the frame, it still encoded a lot more information and then you're processing.

[313] Right.

[314] And if you take a large, it turns out if you take a large number of images of things in general, so you've seen these things where you take a 2D image and turn it into a 3D image.

[315] Yeah.

[316] Right.

[317] The reason that works is even in the 2D image.

[318] The 3D is in the room actually got embedded in that picture in a way.

[319] And if you have the right understanding of how physics and reality works, you can reconstruct the 3D model.

[320] Okay, so this really is.

[321] But you could, you know, an AI scientist may cruise around the world with infrared and radio wave cameras.

[322] And they might take pictures of all different kinds of things.

[323] And every once of all, they'd show up and go, hey, the sun, you know, I've been staring at the sun and the ultraviolet and radio waves for the last month.

[324] And it's way different than anybody thought because humans tend to look at light and visible spectrum.

[325] And, you know, there could be some really novel things coming out of that.

[326] Well, so.

[327] But humans also, we live in the spectrum we live in, because it's a pretty good one for planet Earth.

[328] Like, it wouldn't be obvious that AI would start some different place.

[329] Like, visible spectrum is interesting for a whole bunch of reasons.

[330] Right.

[331] So in a set of images that are human derived, you're saying that there's, the way I would conceptualize that is that there's two kinds of logos embedded in that.

[332] One would be that you could extract out from that set of images what was relevant to human beings.

[333] But you're saying that the fine structure of the objective world, outside of human concern, is also embedded in the set of images, and that an AI system could extract out a representation of the world, but also a representation of what's motivating to human beings.

[334] Yes.

[335] And then some human scientists already do look at the sun and radio waves and other things, because they're trying to, you know, get different angles on how things work.

[336] Yeah, well, I guess it's a curious thing.

[337] It's like the same with like buildings and architecture, mostly fit people.

[338] Well, the other...

[339] There's a reason for that.

[340] The reason why I keep coming back to hammering the same point is that even in terms of the development of the AI, that is developing AI requires immense amount of money, energy, you know, and time.

[341] That's a transient thing.

[342] In 30 years, it won't cost anything.

[343] So that's going to change so fast.

[344] It's amazing.

[345] So that's a, like, supercomputers used to cost millions of dollars, and now your phone is the supercomputer.

[346] So it's the time between millions of dollars and $10 is about 30 years.

[347] So it's like, I'm just saying it's like the time and effort isn't a thing in technology.

[348] It's moving pretty fast.

[349] It's just, that's just, that just says to date.

[350] Yeah.

[351] But even making, even making, let's say, even, I mean, I guess, Maybe this is the nightmare question.

[352] Like, could you imagine an AI system which becomes completely autonomous, which is creating itself even physically through automized factories, which is, you know, programming itself, which is creating its own goals, which is not at all connected to human endeavor?

[353] I mean, individual researchers can, you know, I have a friend who, I'm going to introduce you to him tomorrow, he wrote a program that scraped all of the Internet and trained an A &A.

[354] AI model to be a language model on a relatively small computer.

[355] And in 10 years, the computer he could easily afford would be as smart as a human.

[356] So he could train that pretty easily.

[357] And that model could go on Amazon and buy 100 more of those computers and copy itself.

[358] So, yeah, we're 10 years away from that.

[359] And then why, like, why would it do that?

[360] I mean, what does, is it possible?

[361] It's all about the motivational question.

[362] I think that that's what even Jordan and I both have been coming at from the outset.

[363] It's like, so you have an image, right?

[364] You have an image of Skynet or of the Matrix, you know, in which the sentient AI is actually fighting for its survival.

[365] So it has a survival instinct, which is pushing it to self -perpetuate, like to replicate itself and to create variation on itself in order to survive and identifies humans as an obstacle to that, you know.

[366] Yeah, yeah.

[367] So you have a whole bunch of implicit assumptions there.

[368] So humans, last I checked, are unbelievably competitive.

[369] And when you let people get into power with no checks on them, they typically run amok.

[370] It's been historical experience.

[371] And then humans are, you know, self -regulating to some extent, obviously with some serious outliers because they self -regulate with each other.

[372] And humans and AI models at some point.

[373] will have to find their own calculation of self -regulation and trade -offs about that stuff.

[374] Yeah, because AI doesn't feel pain, at least that we don't know that it feels.

[375] Well, lots of humans don't feel pain either.

[376] So, I mean, that's, I mean, humans feeling pain or not doesn't, you know, doesn't stop a whole bunch of activity.

[377] I mean, that's not.

[378] I mean, it doesn't, the fact that we feel pain doesn't stop.

[379] Doesn't regulate that many people.

[380] Right, right.

[381] I mean, there's definitely people like, you know, children, if you threaten them, you know, go to your room and stuff, you can regulate them that way.

[382] But some kids ignore that completely.

[383] And adults are the same way.

[384] And it's often counterproductive.

[385] Yeah.

[386] So, right, you know, culture and societies and organizations, we regulate each other, you know, sometimes success.

[387] In competition and cooperation.

[388] Yeah.

[389] Do you think that, well, we've talked about this to some degree for decades.

[390] I mean, when you look at how.

[391] fast things are moving now.

[392] And as you push that along, when you look out 10 years and you see the relationship between the AI systems that are being built and human beings, what do you envision?

[393] Or can you envision it?

[394] Well, can I?

[395] Yeah.

[396] Like I said, I'm a computer guy.

[397] And I'm watching this with, let's say, some fascination as well.

[398] I mean, the last, so Ray Kurzweil said, you know, prior to, progress accelerates.

[399] Yeah.

[400] So we have this idea that 20 years of progress is 20 years, but, you know, the last 20 years of progress was 20 years, and the next 20 years will probably be, you know, 5 to 10.

[401] Right, right, right.

[402] And you can really feel that happening.

[403] To some level, that causes social stress independent of whether it's AI or Amazon deliveries.

[404] You know, there's so many things that are going into the stress of it all.

[405] But there's progress, which is an extension of, you know, human capacity.

[406] And then there's this progress which I'm hearing about, the way that you're describing it, which seems to be an inevitable progress towards creating something which is more powerful than you, right?

[407] And so what is that?

[408] I don't even understand that drive.

[409] Like, what is that drive to create something which can supplant you?

[410] So look at the average person in the world, right?

[411] So the average person already exists in this world, because the average person is halfway up the human hierarchy.

[412] There's already many people more powerful than any of us.

[413] They could be smarter, they could be richer, they could be better connected.

[414] We already live in a world.

[415] Like very few people are at the top of anything.

[416] So that's already a thing.

[417] So basically the drive to make someone a superstar, let's say, or the drive to elevate someone above you, that would be the same drive that is bringing us to creating these ultra -powerful machines.

[418] Because we have that.

[419] Like, we have a drive to elevate.

[420] Like, you know, when we see a rock star that we like, people want to submit themselves to that.

[421] They want to dress like them.

[422] They want to raise them up above them as an example, something to follow, right?

[423] Something to subject themselves, too.

[424] You see that with leaders.

[425] You see that in the political world.

[426] And in teams, you see that in sports teams, the same thing.

[427] And so you think that's the drug.

[428] Well, we've always tried to build things that are beyond us.

[429] You know, I mean.

[430] I mean, it's about are we building, are we building?

[431] a god is that what people is that the drive that is pushing someone towards because when i hear what you're describing uh jim i hear something that is extremely dangerous right sounds extremely dangerous to the very existence of humans yet i see humans acting and moving in that direction almost without being able to stop it as if there's no one i think it is unstoppable well that's one of the things we've also talked about is because i've asked jim straight out you know because of the hypothetical danger associated with this, why not stop doing it?

[432] And, well, part of his answer is the ambivalence about the outcome, but also that it isn't obvious at all that in some sense it's stoppable.

[433] I mean, it's the cumulative action of many, many people that are driving this along.

[434] And even if you took out one player, even a key player, the probability that you do anything but slow it infinitesimally is quite...

[435] Yeah, because there's also a massive payoff for those that will succeed.

[436] It's also set up that way.

[437] People know that at least until the AI take over or whatever, that whoever is on the line towards increasing the power of the AI will rake in major rewards.

[438] Right, well, that's what you do with all cognitive acceleration, right?

[439] Yeah, I could recommend Ian Banks as an author, English author, I think.

[440] He wrote a series of books on the, he called The Culture novels.

[441] And it was a world where there was humans and then there was AI as smart as humans and AIs that were dumber than humans, but there were some AIs that were much, much smarter.

[442] And they lived in harmony because they mostly all pursued what they wanted to pursue.

[443] Humans pursued human goals and super smart AIs pursued super smart AI goals.

[444] And they communicated and worked with each other, but they mostly, you know, they're different.

[445] when they were different enough that that was problematic, their goals were different enough that they didn't overlap.

[446] Because one of the things that...

[447] That would be my guess.

[448] It's like these ideas where these super AIs get smart and the first thing they do is stomp out the humans.

[449] It's like, you don't do that.

[450] Like, you don't wake up in the morning and think, I have to stomp out all the cats.

[451] No, it's not about...

[452] The cats do cat things and the ants do ant things and the birds do bird things.

[453] And super smart mathematicians do smart mathematician things.

[454] and, you know, guys who like to build houses, do build house things.

[455] And, you know, everybody, you know, the world, there's so much space in the intellectual zone that people tend to go pursue the, in a good society, like, you tend to pursue the stuff that you do.

[456] And then the people in your zone, you self -regulate.

[457] And you also, even in the social stratas, we self -regulate.

[458] I mean, the recent political events of the last 10 years, the weird thing to me has been why have, you know, people with power have been overreaching to take too much from people with less?

[459] Like, that's bad regulation.

[460] But one of the aspects of - One of the aspects of increase in power is that increase in power is always mediated at least in one aspect by military, by, let's say, physical power.

[461] on others, you know, and we can see that technology is linked and has been linked always to military power.

[462] And so the idea that there could be some AIs that will be our friends or whatever is maybe possible.

[463] But the idea that there will be some AIs, which will be weaponized, seems absolutely inevitable because increasing power is always, increasing technological power always moves towards military.

[464] So we've lived with atomic bombs since the 40s, right?

[465] So the, I mean, the solution to this has been mostly, you know, some form of mutual assured destruction or attacking me, like the response to attacking me is so much worse than the...

[466] Yeah, but it's also because we have reciprocity.

[467] We recognize each other as the same.

[468] So if I look into the face of another human, there's a limit of how a different, I think, that presents.

[469] is for me. But if I'm hearing something described as the possibility of superintelligences that have their own goals, their own cares, their own structures, then how much mirror is there between these two groups of people, these two groups?

[470] Well, Jim's objection seems to be something like we're making, we may be making when we're doomsaying, let's say, and I'm not saying there's no place for that, we're making the presumption of something like a zero sum competitive landscape, right, is that the idea and the idea behind movies like The Terminator is that there is only so much resources and the machines and the human beings would have to fight over it.

[471] And you can see that that could easily be a preposterous assumption.

[472] Now, I think that one of the fundamental points you're making, though, is also there will definitely be people that will weaponize AI.

[473] And those weaponized AI systems will have as their goal, something like the destruction of human beings, at least under some circumstances.

[474] And then there's the possibility that that will get out of control, because the most effective systems at destroying human beings might be the ones that win, let's say.

[475] And that could happen independently of whether or not it is a true zero -sum competition.

[476] Yeah, and also the effectiveness of military stuff doesn't need very smart AI to be a lot better than it is today.

[477] You know, like the Star Wars movies where like, you know, tens of thousands of years in the future, super highly trained, you know, fighters can't hit somebody running across the field.

[478] Like, that's silly, right?

[479] You can already make a gun that can hit everybody in the room without aiming at it.

[480] It's, you know, there's, like, the military threshold is much lower than any intelligence threshold.

[481] like for danger.

[482] And, you know, like to the extent that we self -regulated through the nuclear crisis is interesting.

[483] I don't know if it's because we thought that the Russians were like us.

[484] I kind of suspect the problem was that we thought they weren't like us.

[485] And but we still managed to make some calculation to say that any kind of attack would be mutually devastating.

[486] Well, when you look at...

[487] You know, the destructive power of the military, we are.

[488] have so far exceeds the planet.

[489] I'm not sure, like, adding intelligence to it as the tipping point.

[490] Like, that's, I think the more likely thing is things that are truly smart in different ways will be interested in different things.

[491] And then the possibility for, let's say, mutual flourishing is really interesting.

[492] And I know artists using AI already to do really amazing things, and that's already happening.

[493] Well, when you're working on the frontiers of AI development and you see the development of increasingly intelligent machines, I mean, I know that part of what drives you is, I don't want to put words in your mouth, but what drives intelligent engineers in general, which is to take something that works and make it better, and maybe to make it radically better and radically cheaper.

[494] So there's this drive toward technological improvement, and I know that you like to solve complex problems.

[495] And you do that extraordinarily well.

[496] But do you, is there also a vision of a more abundant form of human flourishing emerging from the, from the development?

[497] So what do you see happening?

[498] Well, you said it years ago.

[499] It's like we're going to run out of energy.

[500] What's next?

[501] We're going to run out of matter.

[502] Right.

[503] Like our ability to do what we want in ways that are interesting and, you know, for some people, beautiful, is limited by a whole bunch of things because where, you know, partly it's technological and partly what's, you know, we're stupidly divisive.

[504] But there is, there's also, there's also a reality, which is one of the things that technology has been is, of course, an increase in power towards desire, towards human desire.

[505] And that is represented in mythological stories where, let's say, technology is used to accomplish impossible desire, right?

[506] have, you know, the story of the story of building the mechanical, the bull around the king of Minos, the wife of the king of Minos, you know, in order to be inseminated by a bull.

[507] We have the story of the, the story of Frankenstein, et cetera, the story of the golem, where we put our desire into this increased power.

[508] And then what happens is that we don't know our desires.

[509] That's one of the things that I've also been worried about in terms of AI.

[510] that we have secret desires that enter into what we do that people aren't totally aware of and as we increase in power these systems those desires let's say the idea for example of the possibility of having an AI friend and the idea that an AI friend would be the best friend you've ever had because that that friend would be the nicest to you would care the most about you would do all those things That would be an exact example of what I'm talking about, which is it's really the story of the genie, right?

[511] It's the story of the genie and the lamp where the genie says, what do you wish?

[512] And I have unlimited power to give it to you.

[513] And so I give him my wish, but that wish has all these underlying implications that I don't understand, all these underlying possibilities that I don't understand.

[514] All these underlying possibilities.

[515] But the cool thing, the moral of almost all those stories is having unlimited wishes will be, lead to your downfall.

[516] And so humans, like, if you give, you know, a young person, an unlimited amount of stuff to drink, for six months they're going to be falling down drunk and they're going to get over it, right?

[517] Having a friend that's always your friend no matter what, it's probably going to get boring pretty much.

[518] Well, the literature on marital stability indicates that.

[519] So there's a sweet spot with regards to marital stability in terms of the ratio of negative to positive communication.

[520] So if, on average, you receive five positive communications and one negative communication from your spouse, that's on the low threshold for stability.

[521] If it's four positive to one negative, you're headed for divorce.

[522] But interestingly enough, on the other end, there's a threshold as well, which is that if it exceeds 11 positive to one negative, you're also moving towards divorce.

[523] so there's so so so there might be self -regulating mechanisms that that would in sense take care of that you might find a yes man AI friend extraordinarily boring very very rapidly but as opposed to an AI friend that was interested in what you were interested and it was actually interesting like you know we go through friends in the course of our lives like different friends are interesting at different times and some friends we grow with and that continues to be really interesting for years and years and years and other friends, you know, some people get stuck in their thing and then you've moved on or they've moved on or something.

[524] So, yeah, I tend to think of a world where there was more abundance and more possibilities and more interesting things to do is an interesting place.

[525] And modern society has let the human population, and some people think this is a bad thing, but I don't know.

[526] I'm a fan of it.

[527] You know, modern population has gone from tens of $200 million to billions of people.

[528] That's generally being a good thing.

[529] We're not running out of space.

[530] I've been in it, you know, some of your audience has probably been in an airplane.

[531] If you look out the window, the country is actually mostly empty.

[532] The oceans are mostly empty.

[533] Like, we're weirdly good at polluting large areas, but as soon as we decide not to, we don't have to.

[534] technology.

[535] Most of our, you know, energy pollution problems are technical.

[536] Like, we can stop polluting.

[537] Like, electric cars are great.

[538] So there's so many things that we could do technically.

[539] I forget the guy's name.

[540] He said the Earth could easily support a population of a trillion people.

[541] And trillion people would be a lot more people doing, you know, random stuff.

[542] And he didn't imagine that the future population would be a trillion humans and a trillion in AIs, but it probably will be.

[543] It will probably exist on multiple planets, which will be good the next time an asteroid shows up.

[544] So what do you think about, so one of the things that seems to be happening, tell me if you think I'm wrong here, and I think it's germane to join.

[545] And I just want to make the point of, you know, where we are compared to living in the Middle Ages, our lives are longer, our families are healthy, our children are more likely to survive, like many, many good things happened.

[546] And, like, sending the clock back wouldn't be good.

[547] And, you know, if we have some care and people who actually care about how culture interacts with technology for the next 50 years, you know, we'll get through this, hopefully, more successful than we did the atomic bomb in the Cold War.

[548] But it's...

[549] Okay, so...

[550] It's a major change.

[551] I mean, this is, like, your worries are, you know, I mean, they're relevant.

[552] But, you know, but also your, Jonathan, your stories about how humans have faced abundance and faced evil kings and evil overlords.

[553] Like, we have thousands of years of history of facing the challenge of the future and the challenge of things that cause radical change.

[554] Yeah.

[555] And, you know, that's very valuable, you know, information.

[556] But for the most part, nobody's succeeded by stopping change.

[557] They've succeeded by...

[558] Bringing to bear on the change, our capability to self -regulate the balance.

[559] Like, a good life isn't having as much gold as possible.

[560] It's a boring life.

[561] A good life is, you know, having some quality friends and doing what you want and having some insight in life.

[562] Yeah.

[563] And some optimal challenge.

[564] And, you know, in a world where a larger percentage of people can have, well, live in relative abundance and have tools.

[565] opportunities, I think is a good thing.

[566] Yeah.

[567] And I don't want to pull back abundance.

[568] But what I have noticed is that our abundance brings a kind of nihilism to people.

[569] And I don't, like I said, I don't want to go back.

[570] I'm happy to live here and to have these tech things.

[571] But I think it's something that I've also noticed, that increase of the capacity to get your desires.

[572] when that increases to a certain extent also leads to a kind of nihilism where exactly that...

[573] Well, I wonder, Jonathan, I wonder if that's partly a consequence of the erroneous maximization of short -term desire.

[574] I mean, one of the things that you might think about that could be dangerous on the AI front is that we optimize the manner in which we interact with our electronic gadgets to capture short -term attention, right?

[575] Because there's a difference between getting what you want right now, right now, and getting what you need in some more mature sense across a reasonable span of time.

[576] And one of the things that does seem to be happening online, and I think it is driven by the development of AI systems, is that we're assaulted by systems that parasitize our short -term attention.

[577] And at the expense of longer -term attention, and I If the AI systems emerge to optimize attentional grip, it isn't obvious to me that they're going to optimize for the attention that works over the medium to long run, right?

[578] They're going to be, they could conceivably maximize something like whim -centered existence.

[579] Yeah, because all the virality is based on that.

[580] All the social media networks are all based on this, on this reduction, this reduction of attention, this reduction of desire to reaching your rest, say in that desire, the like, the click, all these things, they're...

[581] Yeah, now.

[582] Yeah, exactly.

[583] So, but that's something that, you know, so for reasons that are somewhat puzzling, but maybe not, you know, the business models around a lot of those interfaces are around, you know, the part, the user's the product and, you know, the advertisers are trying to get your attention.

[584] Yeah, yeah.

[585] But that's something culture could regulate.

[586] We could decide that, no, we don't, we don't want tech platforms to be.

[587] be driven by advertising money.

[588] Like, that would be a smart decision, probably.

[589] And that could be a big change.

[590] And what do you see as an alter...

[591] Well, the problem that might be that might be that markets drive that in some sense, right?

[592] And I know they're driving that in a shorter way.

[593] But we can take steps.

[594] Like, you know, at various times, you know, alcohol has been illegal.

[595] Like, you can...

[596] Society can decide to regulate all kinds of things.

[597] And, you know, sometimes some things need to be regulated and some things don't.

[598] Like, when you buy a hammer, you don't fight with your hammer for its attention, right?

[599] A hammer's a tool.

[600] You buy one when you need one.

[601] Nobody's marketing hammers to you.

[602] Like, that has a relationship that's transactional to your purpose, right?

[603] Yeah.

[604] And our technology has become a thing where, I mean, in the short run, things are so...

[605] There's a relationship between human, let's say high human goals, something like attention and status.

[606] And what we talked about, which is the idea of elevating something higher in order to see it as a model.

[607] See, these are where intelligence exists in the human person.

[608] So when we notice that in the systems, in the platforms, these are the aspects of intelligence which are being weaponized in some ways, not against us, but are just kind of being weaponized because they're the most beneficial at the short term to be able to generate our constant attention.

[609] And so what I mean is that that is what the AIs are made of, right?

[610] They're made of attention, prioritization, you know, good, bad.

[611] What is it that is worth putting energy into in order to predict towards a telos?

[612] And so I'm seeing that it's the idea that we could disconnect them suddenly seems very difficult to me. Yeah, so I'll give you two.

[613] First I want to give an old example.

[614] So after World War II, America went through this amazing building boom of building suburbs.

[615] And the American dream was, you could have your own house, your own yard in a suburb with a good school, right?

[616] So in the 50s, 60s, early 70s, they were building like crazy.

[617] By the time I grew up, I lived in a suburb in dystopia.

[618] And we found that that as a goal wasn't a good thing because people ended up in houses separated from, social needs and structures, and then new towns are built around like a hub with, you know, places to go and eat, you know.

[619] So there was a good that was viewed in terms of opportunity and abundance, but it actually was a fail culturally.

[620] And then some places it modified and it continues in some places are still dystopian, you know, suburban areas.

[621] And in some places people simply learn to live with it.

[622] So that has something, that has to do with it.

[623] attention, by the way.

[624] It has to do with a subsidiary hierarchy, like a hierarchy of attention, which is set up in a way in which all the levels can have room to exist, let's say.

[625] And so, you know, the new systems, the new way, let's say the new urbanist movement, similar to what you're talking about, that's what they've understood.

[626] It's like we need places of intimacy in terms of the house.

[627] We need places of communion in terms of, you know, parks and alleyways and buildings where we meet and a church, all these places that kind of manifest our community together.

[628] Yeah.

[629] So those existed coherently for long periods of time.

[630] And then the abundance post -World War II and some ideas about like what life could be like caused this big change.

[631] And that chain satisfied some needs, people got houses, but broke community needs.

[632] And then new sets of ideas about what's the sense of this, what's the possibility of having your own home, but also having community, not having to drive 15 minutes for every single thing.

[633] And some people live in those worlds and some people don't.

[634] Do you think we'll be smart?

[635] So one of the problems, why were we smart enough to solve some of those problems?

[636] Because we had 20 years.

[637] But now, because one of the things that's happening now is we're, as you pointed out earlier, is we're going to be producing equally revolutionary transformations, but at a much smaller scale of time.

[638] And so, Mike, one of the things I wonder about, I think it's driving some of the concerns in the conversation is, are we going to be intelligent enough to direct with regulation the transformations of technology as they start to accelerate?

[639] I mean, we've already, look what's happened online.

[640] I mean, we've inadvertently, for example, radically magnified.

[641] the voices of narcissists, psychopaths, and Machiavellians.

[642] And we've done that so intensely, partly, and I would say partly as a consequence of AI mediation, that I think it's destabilizing the entire political economy.

[643] Well, it's just debilizing part of it, like Scott Adams point out, you just block everybody that acts like that.

[644] I don't pay attention to people that talk like that.

[645] Yeah, but they seem to be raising the temperature in the entire culture.

[646] Like, 10 ,000 people here can make a storm in some quote.

[647] corporate person, you know, fire somebody.

[648] But I think that's, like, we're five years from that being over.

[649] Corporation will go 10 ,000 people out of 10 billion.

[650] Not a big deal.

[651] Okay, so you think that's a learning moment that will re -regulate.

[652] What's natural to our children is so different than it was natural to us.

[653] But what was natural to us was very different from our parents.

[654] So some changes get accepted generationally, really fast.

[655] What's made you so optimistic?

[656] What do you mean optimistic?

[657] Well, most of the things that you have said today, and maybe it's also because we're pushing you, I mean, you really do.

[658] My nephew, Kyle, was really a smart, clever guy.

[659] He called me a, what did he call it, a cynical optimist.

[660] Like, I believe in people.

[661] Like, I like people, but also people are complicated.

[662] They all got all kinds of nefarious goals.

[663] I worry a lot more about people burning down the world than I do about artificial intelligence just because, you know, people, well, you know people, they're difficult, right?

[664] But the interesting thing is an aggregate we mostly self -regulate.

[665] And when things change, you have these dislocations, and then it's up to people who talk and think, and while we're having this conversation, I suppose, to talk about how do we re -regulate this stuff.

[666] Yeah.

[667] Well, because one of the things that the increase in power has done in terms of AI, and you can see it with Google and you can see it online, is that there are certain people who hold the keys, let's say, and then who hold the keys to what you see and what you don't see.

[668] So you see that on Google, right?

[669] And you know it if you know what searches to make, where you realize that this is not, this is actually being directed by someone who now has huge amount of power in order to direct my attention towards their ideological purpose.

[670] And so that's why, like, I think that, to me, I always think it would, I always tend to see AI as as an extension of human power, even though there is this idea that it could somehow become totally independent, I still tend to see it as an increase of the human care.

[671] And whoever will be able to hold the keys to that will have increase in power.

[672] And that can be, like, and I think we're already seeing it.

[673] Well, that's not, that's not really any different, though, is it, Jonathan?

[674] The situation that's always confronted us in the past, I mean, we've always had to deal with the evil uncle of the king, and we've always had to deal with the fact that an increase in ability could also produce a commensurate increase in tyrannical power, right?

[675] I mean, so that might be magnified now, and maybe the danger in some sense is more acute, but possibly the possibility is more present as well.

[676] Because you can train the AI to find hate speech, right?

[677] You can train an AI to find hate speech and then to act on that hate speech immediately.

[678] Now it's only, we're not only talking about social media, but what we've seen is that that is now encroaching into payment systems and into people losing their bank account, their access to different services.

[679] And so this idea of automation.

[680] Yeah, there's an Australian bank that already has decided that it's a good thing to send all of their customers a carbon load report every month, right?

[681] And to offer them hints about how they could reduce their polluting purchases, let's say.

[682] And, well, at the moment, that system is one of voluntary compliance, but you can certainly see in a situation like the one we're in now that the line between voluntary compliance and involuntary compulsion is very, very thin.

[683] Yeah.

[684] So I'd like to say, so during the early computer world, computers were very big and expensive.

[685] And then they made many computers and workstations, but they were still corporate only.

[686] And then the PC world came in.

[687] All of a sudden, PCs put everybody online.

[688] Everybody could suddenly see all kinds of stuff.

[689] And, you know, people could get a Freedom Information Act request, put it online somewhere, and 100 ,000 people could see it.

[690] It was an amazing democratization moment.

[691] And then there was a similar, but a smaller revolution with the world of, you know, smartphones and apps.

[692] But then we've had a new, completely different set of companies, by the way, you know, from what happened in the 60s, 70s, 70s, and 80s to today, it's a very different companies that control it.

[693] And there are people who worry that AI will be a winner -take -all thing.

[694] Now, I think so many people are using it and they're working on it so many different places and the cost is going to come down so fast that pretty soon you'll have your own AI app that you'll use to mediate the Internet to strip out the endless stream of ads and you can say, well, is this story objective?

[695] Well, here's the 15 stories and this is being manipulated this way and this is being manipulated that way.

[696] And you can say, well, I want what's more like the real story?

[697] And the funny thing is information that's broadly distributed and has lots of inputs is very hard to fake the whole thing.

[698] So right now a story can pull through a major media outlet, and if they can control the narrative, everybody gets to a fake story.

[699] But if the media is distributed across a billion people who are all interacting in some useful way, somebody standing up.

[700] Yeah, there's real signal there.

[701] And if somebody stands up and says something that's not true, everybody goes, everybody knows that's not true.

[702] So, like, a good outcome with people thinking seriously would be the democratization of information and, you know, objective facts in the same way.

[703] The same thing that happened with PECs versus corporate central computers could happen again.

[704] So you have an increase, the problem is that these are the increase in power.

[705] The increasing power always creates the two at the same time.

[706] And so we saw that, you know, increase in power creates first, or it depends in which direction it happens, it creates an increase in decentralization, an increase in access, and it creates all that.

[707] But then it also at the same time creates the counter reaction, which is an increase in control and increase in centralization.

[708] And so now, the more the power is, the more the waves will, the bigger the waves will be.

[709] And so the image of the image that 1984 presented to us, you know, of people going into newspapers and changing the headlines and taking the pictures out and doing that, that now obviously can happen with just a click.

[710] So you can click and you can change the past.

[711] You can change the past.

[712] You can change facts about the world because they're all held, you know, online.

[713] And we've seen it happen, obviously, in the media recently.

[714] So does decentralization win over centralization?

[715] How is that even possible, it seems?

[716] I mean, and it's also interesting.

[717] Like when Amazon became a platform, suddenly any mom and pop business could have a, you know, Amazon, eBay, there's a bunch of platforms, which had an amazing impact.

[718] Because any business could get to anybody.

[719] But then the platform itself started to control the information flow.

[720] Yeah.

[721] Right.

[722] But at some point, that'll turn into, people go, well, why am I letting somebody control my information flow?

[723] And Amazon objectively doesn't really have any capability.

[724] Right.

[725] So like you point out, the waves are getting bigger, but they're real waves.

[726] It's the same with information.

[727] The information is all online.

[728] It's also on a billion hard drives.

[729] Right.

[730] So somebody says, I'm going to raise the objective fact.

[731] A distributed information system would say, yeah, go ahead and erase it anywhere you want.

[732] There's another thousand copies of it.

[733] Yeah, and that's what that's what the people tried to do, did they?

[734] Yeah, yeah, and this is where thinking people have to say, yeah, this is a serious problem.

[735] Like if humans don't have anything to fight for, they get lazy and, you know, a little bit dopey, in my view.

[736] Like, we do have something to fight for.

[737] And, you know, that's worth, that's worth talking about.

[738] like what would a great world with you know distributed you know in human intelligence and artificial intelligence working together in a collaborative way to create abundance and fairness and you know like some some better way at arriving at good decisions than what the truth is that would be a good thing but you know it's not well we'll leave it to the experts and then the experts will tell us what to do that's a bad thing yeah so that's well so do you it's it's a more model that you just laid out, which I think is very interesting.

[739] I'm not somewhat optimistic about that.

[740] Well, it did happen on the computational front.

[741] I mean, it was the case.

[742] It happened a couple times both directions.

[743] Okay.

[744] Right?

[745] You know, the PC revolution was amazing.

[746] Yeah.

[747] Right.

[748] And Microsoft was a fantastic company.

[749] It enabled everybody to write a $10, $50 program to use.

[750] Yeah.

[751] Right.

[752] And then at some point, they're also, you know, let's say a difficult company.

[753] And they made money off a lot of people and became extremely valuable.

[754] Now, for the most part, they haven't been that directional and telling you what to do and think and how to do it.

[755] But they are a many -making company.

[756] You know, Apple created the app store, which is great.

[757] But then they also take 30 % of the up -store profits, and there's a whole section of the Internet that's fighting with Apple about their control of that platform.

[758] And in Europe, you know, they've decided to regulate some of that, which that should be a conversation.

[759] that should be a social, cultural conversation about how should that work.

[760] Yeah.

[761] So do you see the more likely, certainly the more desirable future, is something like a set of distributed AIs, many of which are under personal, in personal relationship in some sense, the same way that we're in personal relationship with our phones and our computers, and that that would give people the chance to fight back, so to speak against this.

[762] And there's lots of people really interested in distributed platforms.

[763] And one of the interesting thing about the AI world is, you know, there's a company called Open AI, and they open source a lot of it.

[764] The AI research is amazingly open.

[765] It's all done in public.

[766] People publish the new model all the time.

[767] You can try them out.

[768] People, there's a lot of startups doing AI in all different kinds of places.

[769] You know, it's a very curious phenomenon.

[770] Yeah.

[771] And it's kind of like a big, huge wave.

[772] It's not like a, you can't stop a wave with your hand.

[773] Yeah.

[774] Well, when you think about waves, there are two actually in the book of Revelation, which describes the end or describes the finality of all things or the totality of all things is maybe a way for people who are more secular to kind of understand it.

[775] And in that book, there are two images, interesting images about technology.

[776] One is that there is a dragon that falls from the heavens.

[777] and that dragon makes a beast, and then that beast makes an image of the beast, and then the image speaks.

[778] And when the image speaks, then people are so mesmerized by the speaking image that they worship the beast ultimately.

[779] So that is one image of, let's say, making and technology in Scripture in Revelation.

[780] But there's another image, which is the image of the heavenly Jerusalem, and that image is more an image of balance.

[781] It's an image of the city which comes down from heaven with a garden in the center and then becomes this glorious city.

[782] And it says the glory of all the kings is gathered into the city.

[783] So the glory of all the nations is gathered into this city.

[784] So now you see a technology which is at the service of human flourishing and takes the best of humans and brings it into itself in order to kind of manifest.

[785] And it also has hierarchy, which means it has the natural at the center and then has the artificial as serving the natural, you could say.

[786] So those two images seem to reflect these two waves that we see and this kind of idea of an artificial intelligence which will be ruling over us or speaking over us.

[787] But there's a secret person controlling it, even in Revelation.

[788] It's like there's a beast controlling it and making it speak.

[789] So now we're mesmerized by it.

[790] And then this other image.

[791] So I don't, I don't, Jordan, if you ever thought about those two images in Revelation as being related to technology, let's say.

[792] Well, I don't think I've thought about those two images in the specific manner that you described, but I would say that the work that I've been doing, and I think the work you've been doing too, and the public front reflects the dichotomy between those images, and it's relevant to the points that Jim has been making.

[793] I mean, we are definitely increasing our technological power, and you can imagine that that will increase our capacity for tyranny and also our capacity for abundance.

[794] And then the question becomes, what do we need to do in order to increase the probability that we tilt the future towards Jerusalem and away from the beast?

[795] And the reason that I've been concentrating on helping people bolster their individual morality to the degree that I've managed that is because I think that whether the outcome is the positive outcome that in some sense Jim has been outlining or the negative outcomes that we've been querying him about, I think that's going to be dependent on the individual ethical choices of people, well, at the individual level, but then cumulatively, right?

[796] So if we decide that we're going to worship the image of the beast, so to speak, because we're mesmerized by our own reflection, that's another way of thinking about it.

[797] And we want to be the victim of our own dark desires, then the IAA revolution is going to go very, very badly.

[798] But if we decide that we're going to aim up in some positive way and we make the right micro -decisions, well, then maybe we can harness this technology to produce a time of abundance in the manner that Jim is hopeful about.

[799] Yeah, and let me make two funny points.

[800] So one is, I think there's going to be continuum.

[801] Like the word artificial intelligence won't actually make any sense.

[802] Right.

[803] So humans collectively, like individuals know stuff, but collectively we know a lot more, right?

[804] And the thing that's really good is in a diverse society with lots of people pursuing.

[805] individual, interesting, you know, ideas, worlds.

[806] Like, we have a lot of things.

[807] And more people, more independence, generates more diversity.

[808] And that's a good thing where, you know, it's a totalitarian society where everybody's told to wear the same shirt.

[809] And, like, it's inherently boring.

[810] Like, the beast speaking through the monster is inherently dull, right?

[811] But in an intelligent world where not only can we have more intelligent things, but in some places, go far beyond what most humans are capable of, in pursuit of interesting variety.

[812] I believe the information, well, let's say intelligence is essentially unlimited, right?

[813] Like, and the unlimited intelligence won't be this shiny thing that tells everybody what to do.

[814] That's sort of the opposite of interesting intelligence.

[815] Interesting intelligence will be more diverse, not less diverse.

[816] Like, that's a good future.

[817] And your second description, that seems like a future was working for and also worth fighting for.

[818] And that means concrete things today.

[819] And also, you know, it's a good conceptualization.

[820] Like, I see the messages.

[821] My kids are taught, you know, don't have children, and the world's going to end.

[822] We're going to run out of everything.

[823] You're a bad person.

[824] Why do you even exist?

[825] It's like, these messages are terrible.

[826] The opposite is true.

[827] More people would be better.

[828] We live in a world of potential abundance, right?

[829] It's right in front of us.

[830] Like, there's so much energy available.

[831] It's just amazing.

[832] It's possible to build technology without, you know, pollution consequences.

[833] That's called externalizing costs.

[834] Like, we know how to do that.

[835] We can have very good, clean technology.

[836] We can do, it would do lots of interesting things.

[837] So if the goal is maximum diversity, then the line between human intelligence, artificial intelligence that we draw, like, you'll see all these kind of really interesting partnerships and all kinds of things and more people doing what they want, which is the world I want to live in.

[838] Yeah.

[839] But to me, it seems like the question is, going to be related to attention ultimately.

[840] That is, what are humans attending to it the highest?

[841] What is it that humans care for in the highest?

[842] In some ways, you could say, what do humans, what are humans worshipping?

[843] And like, depending on what humans worship, then their actions will play out in the technology that they're creating, in the increase in power that they're creating.

[844] Well, that's, well, and if we're guided by the negative vision, the sort of thing that Jim laid out that is being talked to his children, can imagine that we're in for a pretty damn dismal future, right?

[845] Human beings are a cancer on the face of the planet.

[846] There's too many of us.

[847] We have to accept top -down, compelled limits to growth.

[848] There's not enough for everybody.

[849] A bunch of us have to go because there are too many people on the planet.

[850] We have to raise up the price of energy so that we don't burn the planet up with carbon dioxide pollution, etc. It's a pretty damn dismal view of the potential that's in front of us.

[851] And so— Yeah, the world should be exciting, and the future should be exciting.

[852] Well, we've been sitting here for about 90 minutes, bandying back and forth both visions of abundance and visions of apocalypse.

[853] And I mean, I've been hardened, I would say, over the decades talking to Jim about what he's doing on the technological front.

[854] And I think part of the reason I've been heartened is because I do think that his vision is guided primarily by desire to help bring about.

[855] something approximating life more abundant.

[856] And I would rather see people on the AI front who were guided by that vision working on this technology.

[857] But I also think it's useful to do what you and I have been doing in this conversation, Jonathan, and acting in some sense as friendly critics and hopefully learning something in the interim.

[858] Do you have anything you want to say in conclusion?

[859] I mean, I just think that the question is linked very directly to what we've been talking about now for several years, which is the question of attention.

[860] the question of what is the highest attention.

[861] And I think the reason why I have more alarm, let's say, than Jim, is that I've noticed that in some ways human beings have come to now, let's say, worship their own desires.

[862] They've come to worship.

[863] And that even the strange thing of worshipping their own desires is that actually led to an anti -human narrative.

[864] You know, this is a weird idea, it's almost suicidal desire that humans have.

[865] And so I think that seeing all of that together in the increase of power, I do worry that the image of the beast is closer to what will manifest itself.

[866] And I feel like during COVID, that sense in me was accelerated tenfold in noticing to what extent technology was used, especially in Canada, how technology was used to instigate something which looked like authoritarian systems.

[867] And so I am worried about it.

[868] But I think, like Jim, honestly, although I say that, I do believe that in the end, wins.

[869] I do believe that in the end, you know, these things will level themselves out.

[870] But I think that because I see people rushing towards AI almost, you know, almost like lemmings going off a cliff, I feel like it is important to sound the alarm once in a while and say, you know, we need to orient our desire before we go toward this extreme power.

[871] So I think that that's mostly the thing that worries me the most and that preoccupies me the most.

[872] But I think that, ultimately in the end, I do share Jim's positive vision.

[873] And I do think that, I do believe the story has a happy ending.

[874] It's just, you might have to go through hell before we get there.

[875] I hope not.

[876] So, Jim, how about you, what have you got to say in closing?

[877] A couple years ago, a friend who's, you know, my age, said, oh, kids coming out of college, they don't know anything anymore.

[878] They're lazy.

[879] And I thought, I work at Tesla.

[880] I was working at Tesla at the time.

[881] And we hired kids out of college, and they couldn't wait to make.

[882] things.

[883] They were like, it's a hands -on place, it's a great place.

[884] And I've told people, like, if you're not in a place where you're doing stuff, it's growing, it's making things.

[885] You need to go somewhere else.

[886] And also, I think you're right, the mindset of, if people are feeling this is a productive, creative technology, that's really cool, they're going to go build cool stuff.

[887] And if they think it's a shitty job and they're just tuning the algorithm so they can get more clicks, They're going to make something beastly, you know, beastly, perhaps.

[888] And the stories, you know, our cultural tradition is super useful, both cautionary and, you know, explanatory about something good.

[889] And I think it's up to us to go do something about this.

[890] And I know people are working really hard to make, you know, the Internet a more open place that make sure information is distributed to make sure AI isn't a winner, take -all thing like these are real things and people should be talking about them and then they should be worrying but the the upside's really high and we've faced these kind of technological like this is a big change like AI is bigger than the internet like I've said this publicly like the internet was pretty big and you know this is bigger it's true but the possibilities are amazing.

[891] And so with some sense, we could utilize them.

[892] Yeah, with some sense we could achieve it.

[893] Yeah.

[894] And the world is interesting.

[895] Like, I think it'll be a more interesting place.

[896] Well, that's an extraordinarily cynically optimistic place to end.

[897] I'd like to thank everybody who is watching and listening.

[898] And thank you, Jonathan, for participating in the conversation.

[899] It's much appreciated as always.

[900] I'm going to talk to Jim Keller for another half an hour on the Daily Wire Plus platform, I use that extra half an hour to usually walk people through their biography.

[901] I'm very interested in how people develop successful careers and lives and how their destiny unfolded in front of them.

[902] And so for all of those of you who are watching and listening, who might be interested in that, consider heading over to the Daily Wire Plus platform and partaking in that.

[903] And otherwise, Jonathan, we'll see you in Miami in a month and a half to finish up the Exodus seminar.

[904] We're going to release the first half of the Exodus seminar we recorded Miami on November 25th, by the way.

[905] So that looks like it's in the can.

[906] Yeah, I can't work with you.

[907] The rest of you?

[908] Yeah.

[909] Yeah, yeah, absolutely.

[910] I'm really excited about it.

[911] And just for everyone watching and listening, I brought a group of scholars together about two and a half months ago.

[912] We spent a week in Miami, some of the smartest people I could gather around me, to walk through the book of Exodus.

[913] We only got through halfway because it turns out there's more information there than I had originally considered, but it went exceptionally well, and I learned a lot.

[914] And Exodus means ex -Hodos.

[915] That means the way forward.

[916] And, well, that's very much relevant to everyone today as we strive to find our way forward through all these complex issues, such as the ones we were talking about today.

[917] So I would also encourage people to check that out when it launches on November 25th.

[918] I learned more in that seminar than any seminar I ever took in my life, I would say.

[919] So it was good to see you there.

[920] We'll see you in a month and a half.

[921] Jim, we're going to talk a little bit more on the Daily Wear Plus platform.

[922] And I'm looking forward to meeting the rest of the people in your AI -oriented community tomorrow and learning more about, well, what seems to be an optimistic version of a life more abundant.

[923] And to all of you watching and listening, thank you very much.

[924] Your attention isn't taken for granted and it's much appreciated.

[925] Hello everyone, I would encourage you to continue listening to my conversation with my guest on dailywireplus .com.