Will AI bring us together or drive us apart? — Independently compared extended review
Mohammad Hassan: There's no turning back. AI is everywhere, from maps to shopping. Some even get therapy from chatbots. Yet they rarely ask whose expertise those bots reflect. So AI shape our collective thinking, our culture? AI models are said to carry the values of those who have created them. Some have been called Some woke. Others appear to change from day to day or even user to user.
I'm Mohammad Hassan and this is the Doha Debates podcast.
Speaker C (context; identity uncertain): We should be asking is, To what extent AI will solidify and dependence that our society already has on corporations.
James Brusseau: I think it's the versus him or herself. That's where the struggle for freedom will exist in the future.
Speaker E (context; identity uncertain): Even though these models are supposed to be open for use they ultimately come embedded with and political ideologies.
Speaker F (context; identity uncertain): This idea that there's this existential race between the US and China to get to superintelligence first this is like a path to
Speaker B (context; identity uncertain): Suicide.
Today's debate: we heading into a world divided by AI tribes? And is this something to celebrate or fear?
Mohammad Hassan: I'd like to welcome our guests. James Brusseau is a professor of philosophy and computer science at Pace University. Robert Wright is the author of The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning. Elina Noor is a senior fellow in the Asia programme at Carnegie Endowment for International Peace. And Evgeny Morozov is a tech theorist and author. A warm welcome to you all.
Multiple speakers (context; identities uncertain): - Thank you. - Happy to be here.
Mohammad Hassan: The four of you are experts in this topic. And me, like a lot of people is just trying to figure out what is going on. So I want us to have a rigorous high level debate. But at the same time, I'm going to interject from time to time with some really dumb questions. So please allow me.
James, if we can begin with you can you give us a picture of the AI landscape as it exists today? Do you think the direction we're heading in is one of unity or division?
James Brusseau: Well, the landscape is certainly confused. All of us, I suspect are somewhat confused about what we see happening around us.
But to my mind, the landscape is quite full of potential. So it seems to me that there are many kinds of positive aspects to the changes we are seeing. There are different ways that we'll be able to receive and understand information. So my general outlook is positive.
I will say quickly. It seems to me that the major threat that we are facing now is one of a kind of homogenisation. That is, I'm thinking just of the experience all of us have had. For example, on Netflix, when we see a movie and then enjoy it and we get recommended another which is similar and then another, which is similar. I think the same thing goes for LinkedIn which is proposing job opportunities for us or for dating websites, which is proposing partners for us. These kinds of tools increasingly force us to accept possibilities which correspond with the data that we have accumulated about our own past. Right? That's how Netflix produces your movie recommendations. They say, Well, what have you watched in the past? What have you enjoyed? This is what you're going to get.
Now that can be very satisfying and enjoyable but the risk ... And so to answer the question I've said there are some positive things. The risk and what worries me about this technology is the possibility for homogenisation even with ourselves that is, it will become impossible for us to escape our own data. The preferences and interests that we have established about ourselves will determine what we see when we turn on the television set. What jobs we are offered, what partners we might receive. So I think that is the threat.
And then finally, I will add, I think that the challenge that waits for us is to find ways to use these tools to help us escape who we are to help us become new and different kinds of people. And I think that if we can do that, then I believe that the future will be very positive for this technology and for humanity.
Mohammad Hassan: Elina, if AI is creating a homogenous experience of the way that we see the world, the way we interact with technology is that not then going to lead to a kind of unity?
Speaker E (context; identity uncertain): Yes, potentially. And I think it depends again on the use cases. James pointed out some examples. I think one of the ... examples that has a double-edged sword is these accent neutralisation programmes that are being churned out by AI. So for example, you have all these business processing organisations outfits in places like the Philippines and in India and Kenya. And on the one hand, they allow for greater work opportunities. This idea that there is dignity with work. At the same time, through this accent neutralisation software it flattens the accents around the world in order for there to be greater efficiency greater levels of communication, supposedly.
But what does it do to a person's identity a person's culture? The way they speak reflects their history, their experiences. And once you start flattening that yes, there is some unity of communication, I suppose it facilitates easy listening and understanding. But at the same time, I think there's a real risk of people losing who they are at the risk of facilitating communication and understanding.
Mohammad Hassan: Evgeny, what do you think about this idea that our identities themselves are being shaped by this AI or are we being made to become robots that all think and behave the same way?
Speaker C (context; identity uncertain): Our identities are shaped by almost anything we use by any kind of technological or ideological infrastructure. So of course, AI would be just like any other technology exercising effects upon people who use it. The question is, What kind of effects do we want it to exercise upon us in the long term? And what kind of options do we still have in terms of determining the path and the trajectory in which AI systems would travel? Because if we look at earlier technologies we see a very interesting trend. All of them start as a series of competing ideologies about the good life. What it means to live with the aeroplane what it means to live with the car or what it means to live with cable television. Those ideologies, however, all get embedded into socioeconomic systems of various kinds into governance, into corporations into regulatory bodies. And in some cases, there is some resonance where a certain ideology for individualistic driving or living gets attached to a set of business interests and it crowds out all the others.
And in some cases, like the United States this car-centric ideology won out. In some cases, like in many European countries, it did not where you still have the idea of public transport. I don't think that AI is any different. We are still very early in its development cycle and there are still relatively open pathways that it can travel depending on what kind of ideology is allowed to win out and what kind of socioeconomic institutions are in place to be able to balance what would otherwise be the dominant ideology which in our day is one that favours the interests of corporations. So whether we manage to balance it out would, to a large extent, determine how homogenising and unifying these technologies would be because they can have all sorts of effects
Speaker uncertain / overlapping speakers (context): And it's still up to us to determine
Mohammad Hassan: What kind of effects we want it to have. And Robert, what do you think of this idea of ideologies? Obviously, this is a really astute point that the technologists who shape our technologies their ideologies, can and will be imprinted onto these systems. Do you think this is what we're seeing already now with AI?
Speaker F (context): I think the people who build the AI's can, in principle, use them
Speaker F (context; identity uncertain): To shape ideologies through the selection of training data through the selection of the experts who play a role in the post training phase. A lot depends on which magazines of which ideological slant are part of the training data.
But even beyond that, I think there may be a naturally you could say, tribalising effect of AI much as there has been with social media and for somewhat the same reason, which is that there's an incentive to maximise engagement. That's what the corporations want. They want you to keep using their product. We've already seen that in the case of social media. That leads to algorithms that reinforce people's conviction that their ideology is right and the other was wrong and feeds them information selectively. And we're already seeing the same thing in AI.
And if anything, AI can in principle do that in a more kind of microscopic, fine-grained way. And we've seen cases of people's very specific, narrow beliefs being reinforced even in a pathological way. So people who think maybe they've discovered some secret formula being encouraged by the AI to believe that even though it's not true
Speaker uncertain / overlapping speakers (context): Or to become more convinced that their conspiracy theories are true.
Multiple speakers (context; identities uncertain): - So psychosis, I think. - Yeah, yeah.
Speaker uncertain / overlapping speakers (context): And it's a real danger
Speaker F (context; identity uncertain): That it will build kind of cocoons for people to live in that are even narrower than our social media cocoons.
Mohammad Hassan: - James, what do you think? - Yeah.
James Brusseau: Let me try two different ideologies on for size here. One ideology is that sort of Heideggerian early 20th century ideology which is that what we are looking for as human beings is some kind of authenticity that is, some sense of who we are which is stable and solid and which we can locate and cling to.
If we want to go that route, if authenticity is the ideology we want to follow, then it seems to me that some of these problems, which you're certainly right to point out, Robert—but some of these problems are symptoms of a kind of strength: that the AI does help us find for ourselves, each one of us, who we are, in some ways that are trivial, going back to my example of Netflix movies, but in some ways which are not trivial. For example, if the AI consistently finds for me on a dating website the same kind of partner then perhaps that's who I am and I should cling to that, hold on to that.
That's one ideology, and I think AI can serve that ideology. But there's also a different possible ideology which is that freedom is more important than authenticity. And in that case, what we would want our artificial intelligence to do and I personally, I will say, I argue for this. What we would want our artificial intelligence to do going back to your point about languages is precisely break those kinds of uniformities those kinds of generic realities that AI can sometimes force us into and help us discover new ways of living and existing. So I think those two ideologies are fundamentally ... It's one way of thinking about this. Fundamentally, what is in play in artificial intelligence is whether we want to be who we are and AI will help us find that or we want the ability to choose different kinds of futures for ourselves ones that are unknown to who we are now.
And the question there is, Can AI help us find that? I think that's a greater challenge because of the way AI produces knowledge. As we said, it produces knowledge by looking at our past and then puts it in front of us as our future again. So can we escape that with artificial intelligence? Potentially. But that would be a long discussion if I went on from there. Elina, what do you think?
Speaker E (context; identity uncertain): So that's a really interesting tension that you brought up. And I think there's an overlap as well between authenticity and freedom because in finding authenticity, you need that freedom as well.
Whether AI can help us find who we really are I think is a more questionable proposition simply because a lot of the systems are built in Global North countries and are sold to us in the global majority. And a lot of that data as well has a bias towards English language data training sets as well as the ideologies that go with that.
So for example in Indonesia Thomas Stamford Raffles was once the administrator of Java and he wrote the book, The History of Java. Now, I'm willing to bet, I don't speak Javanese but I'm willing to bet that his translation and interpretation of the Javanese people is slightly, maybe even dramatically different from how the Javanese would describe themselves. But he, of course, had a different objective to writing the history of Java and it was to tell London at the time that look, these Javanese people they are of a certain civilisation. They are civilised because they have hierarchy in their structure. And this is why perhaps we should not impose such a harsh rule over them. And I don't think the Javanese saw themselves that way.
And so this idea of who we are authentically is partly in that search for freedom but I don't think at this point in time, at this developmental stage understanding full well Evgeny's point that we can shape it. Rght now as it is, given the bias in the training data and the systems of deployment it's not particularly favourable to people in the global majority to find that authenticity when there are narratives about them that have been told by others and that continue to float around in the aether.
James Brusseau: OK. Can I just ask you one quick question? Just a quick thought. Do you think AI is fundamentally about societies and communities or do you think it's fundamentally about individuals in their lives? Because, you see, I was sort of talking about ...
Multiple speakers (context; identities uncertain): - That's a good question. - It is.
Speaker uncertain / overlapping speakers (context): And I assume each of you will have a different idea. And I think that you're coming at this from how societies are.
James Brusseau: So I wonder what you think about that about what AI is, where it goes fundamentally.
Speaker E (context; identity uncertain): Yeah, my instinct is that it's a tool for society. But obviously the emphasis on whether it is something for individuals or for a community per se it's going to depend on the context in which it is deployed.
Speaker uncertain / overlapping speakers (context): Evgeny, what do you think about that? I just want to make a provocation maybe
Speaker C (context; identity uncertain): In that I think that AI in itself does nothing. It's an empty concept, which is up to us to fill with meaning. So in that sense, if you really want to stick to the dichotomy of authenticity versus freedom then we should push it to the ultimate conclusions meaning that we should have the freedom to fill in that concept with the meaning that all of us as individuals or communities would like to fill that with, right? And in that sense, I don't know why we would opt for freedom for us as individuals but we would accept that there is some kind of essentialism or authenticity about AI, that it should be about individuals or it should be about communities. It should not be about anything. It's essentially a very open ended plastic set of technologies which, depending on billions of dollars that are now going to flow into this might go one direction or it might go in a completely different direction.
We've seen that with cable television, we've seen it with the internet we've seen it with a lot of other technologies where there was a lot of rhetoric about how these technologies will develop on their own because they have some inherent properties but the institutional support was not in place and they ended up following the path of least resistance which was an agenda favourable to companies that were well positioned to take advantage of those technologies and monetise them. And the same thing will happen to AI if we assume that its features are God-given that they are to be found in nature the way we found mountains in nature. None of that is like that. It's all up to us and up to institutions, communities, politics and political struggles to define how all of these ingredients should fit together and who should pay the bills.
Should it be individuals paying a monthly subscription fee? Should it be some kind of an intermediary institution like we had with libraries, which did create very utopian spaces which suspended copyright law temporarily so that people can have wider access to literature. Why aren't we thinking about alternative institutional models? Why are we so keen to understand what effects the so-called AI will have? If AI itself is so plastic, that can be anything, right? And this is the part that I think we should leave open
Speaker B (context; identity uncertain): Because ultimately, this is where the true dialectics of AI lies.
Mohammad Hassan: Robert, what do you think of this idea of the plasticity of AI? Do you think this is ... Would you agree with it؟
Speaker uncertain / overlapping speakers (context): In principle, AI can be anything we want in a couple of senses.
Speaker F (context; identity uncertain): First of all, in principle, societies, nations even the whole world can get together and declare policies and constrain it in that way. Moreover, individuals can choose models they like with specific features they like. I don't mean choosing among just the several big proprietary models in the US. That in itself isn't a very broad range of choice but there are other ways to get choice. There are open source models that nations in particular may choose to use to customise them.
And there will be different kind of personas from a given model that people may choose. I'm guessing you'll see versions of ChatGPT that have this characteristic ... You know in principle, you could offer consumers a version of the model that will make them mindful which I'd like. I'd like to be more mindful than I am. You know, it's not easy. It wouldn't surprise me if you have different religions offering different kinds of models.
So in principle, the range is unlimited but I don't think we should act as if it's going to be easy to secure our freedom in the face of AI because the makers of the AI are going to have their own agendas. In the case of corporations, again, that's to keep us using the AI and AI are going to be better and better at manipulating us at figuring out what we like what will make us come back to the AI? I mean, there's already been a study showing that AI's were better than people at persuading other people to change their minds about things. And in particular, the AI's were better if they were given access to personal data about the people they were trying to persuade then they really excelled. OK? And when you remember that an AI can, in principle, search your whole social media history immediately come to know you very well and then set out to persuade you of something like to keep using it like to push whatever buttons of yours will keep you using it you know, that's a lot of power. And I think if we're going to remain free amid the AI revolution we're going to have to try consciously to cultivate what some people are calling cognitive sovereignty and be able to resist manipulative tendencies that I think inevitably will be emanating from the technology.
Mohammad Hassan: James, this question of freedom is really key here. How do we know that we can trust these systems that we're using? Terrifically provocative.
James Brusseau: It seems to me that this is a critical point. In the past, we have been enslaved we have been limited we've been restrained by others by other people, by other institutions. But today, the restraint, the limitation even in some cases, the slavery, in the sense of nudges. All of that is something that we do to ourselves. We are controlled by our own data, our own past. That's what the AI is manipulating. We cannot be nudged, I cannot be nudged by her data or by your data. What the AI uses to control what charity I want to donate to or what person I want to find to date ... What the AI uses to control are my own preferences and my own interests. So it's a very different notion of freedom we have now. Freedom is a struggle that I have with myself and with my own past as opposed to what it used to be a struggle I had with institutions or with others.
So then I think I stand a little bit contrary to your view. It seems to me that we're entering a post-institutional age. And I think that, for the reasons that you were stating everyone can now or very soon will be able to form their own language model which is tailored to their own preferences and their own aspirations.
In the case of language, there are many people working now as you know, of course, better than I do many people working now to create new models which do reflect, say, the Javanese language and culture and so on. So we're going from one general language to specific languages and I think we'll continue moving down toward one person. Each of us will have our own large language model with our own preferences, our own interests.
So I don't think it's the individual versus the institution I think it's the individual versus him or herself. That's where the struggle for freedom will exist in the future.
Mohammad Hassan: Evgeny, is AI in your mind liberating us from these power structures?
Speaker uncertain / overlapping speakers (context): No, no. First of all, I don't agree that we are entering
Speaker C (context; identity uncertain): Any post-institutional landscape. I think we are actually living in a hyper-institutional landscape and the institutions that primarily dominate our lives are corporations and one mad institution that I would call capital. And the data that AI essentially rides on is extracted from me, not because I make these decisions but because there are institutional pressures by banks by financial companies, by the fact that they need to have a credit history by the fact that our parallel welfare system of some kind expects that I need to be using digital services because they're heavily subsidised by venture capital or sovereign wealth funds. I mean, essentially, this is a parallel welfare state that the venture capital industry has been funding for 10 or 15 years which of course results in me giving up a lot of my data.
Speaker uncertain / overlapping speakers (context): But not voluntarily.
Speaker C (context; identity uncertain): No, I mean, those are institutional pressures and it would be naive of us to say that the pressures on a billionaire in Silicon Valley who can be off grid and send their kids to a school where there are no devices are the same as on someone who is earning minimum wage and has to juggle all these apps to essentially stay afloat. Of course, these pressures are very different and that pressure comes from abstract institutions, maybe. Maybe it's the wage that's exercising ... or the fact that you have a stomach and you need to eat. But those pressures are real, they're not fake. So we cannot do anything we want just because there is this suddenly a series of AI apps that allow us to be hyper productive. There are still constraints and pressures. So in that sense, I don't see us liberating ourselves from institutions any time soon.
But beyond that, again, the question that we should be asking is to what extent AI will solidify and accelerate the dependence that our society already has on corporations? We used to have institutions like museums, libraries media and so forth that had some independence from both the state at least theoretically, and the corporations. Right now, if you look at what's left of those institutions they by and large, survive on donations from tech companies and many of them have been replaced by them entirely. And in that sense, AI will accelerate that process and we will end up in a world where the previous setup where at least there was some pluralism and the opposing logic of those institutions was clashing with each other and allowed some space for the individuals and citizens to flourish will just be oriented towards serving the needs of one sector and that sector will be big tech, and that's what worries me.
But again, it's not because AI is inherently favouring that agenda it's because there are already certain processes in place which, unless we oppose them and form coalitions to resist are going to hyper power the corporations
Speaker uncertain / overlapping speakers (context): That have already got quite enough of it in the last 10 or 15 years.
Mohammad Hassan: James, how do you respond to that?
James Brusseau: Yeah, just very quickly. To me that runs a little bit counter to my experience as a person. My experience as a person is that because of artificial intelligence now we see that individuals can make their own movies. I mean, it used to be that if someone had an idea
Multiple speakers (context; identities uncertain): - Like ... - No, you need OpenAI and Nvidia
Speaker C (context; identity uncertain): And all sorts of institutions in the back. It's not individuals
Multiple speakers (context; identities uncertain): - Making their own movies. - So, you're still reliant
Speaker uncertain / overlapping speakers (context): On these systems.
James Brusseau: That's true. But it's nevertheless the case that 20 years ago if I wanted to make a documentary about my philosophical ideas, etc. I needed to go through 30,000 gatekeepers I needed to get credits from one person, a loan from someone else. Now, to use the old phrase, I can sit in my pyjamas in my bedroom and I can produce a fairly capable documentary that might be interesting to some people, about what I think let's say about artificial intelligence and individuality.
So I think in terms of my human experience well, of course you're right. There's always going to be some individual and some structure involved. In terms of my human experience it seems to me that we are entering a phase of very potential individualism that does break through to a significant degree not entirely, but significantly. This power that institutions like Hollywood had over filmmakers Hollywood doesn't have that power anymore.
Multiple speakers (context; identities uncertain): - Similarly ... - How many of those people
Speaker uncertain / overlapping speakers (context): Make a living with the movies that they make on ChatGPT?
Speaker C (context; identity uncertain): How many of those people actually can claim that they are earning money with that?
James Brusseau: I think we are starting to see that on YouTube now.
Similarly, with respect to music we have now developed AI tools which allow individuals to play music with an artificial intelligence complement which actually produces interesting sounds. This is called the Internet of Musical things and that here too, there are possibilities for individuals to explore new... -
Speaker uncertain / overlapping speakers (context): Is it the best possibility that they can have? I mean, this is the wrong trade off.
Speaker C (context; identity uncertain): The real trade off and the real decision that we should be having is can we build a series of institutions that would truly unleash our creativity? Is the current setup we have the best or the worst? I would argue it's the worst. Given what is out there in terms of how much creative infrastructure there is, the system where we all depend on corporations to whom we also have to pay for the most advanced models, very often hundreds of dollars a month—if you want to have access to three or four, Claude, Anthropic ... if you want to get access to Claude, ChatGPT and Gemini, to the top models, as an individual you'll end up spending probably 600 or 700 dollars a month, right? Or would we be better off with a system where this is treated as a public collective good where there are gradients of access where there are institutions that actually want to nurture you not to give you this cheap ability so you can go and click a button and it generates a movie. But that you actually are taught to think structurally ... What it means to build a scene ... How do you build tension? How do you build intrigue? What it means to write a script. All of those things. You can, of course, teach now and maybe we'll end up with a new series of very excellent filmmakers and we'll end up with the renaissance of the film industry. But I just don't trust a bunch of people in Silicon Valley to deliver on that artistic promise. And I'm not denying that AI, as it develops now has the capacity for that but it needs to be tied to an institutional agenda which Silicon Valley is unable to develop.
Speaker uncertain / overlapping speakers (context): That it doesn't want to develop and doesn't have an incentive to ... I don't think the big constraint on creativity per se
Speaker F (context; identity uncertain): Is going to come from the AI. Within a year or two there will be open source models that can do everything in terms of video generation that the best proprietary models can do now. So even if Google or OpenAI wanted to shut down your creative opportunities you would have alternatives. I think the big constraint comes from the platforms. You mentioned YouTube. YouTube doesn't show anything that's uploaded. They have rules. All the platforms have rules. Right now that is where the power is exercised in constraining expression. And I think for some time to come that's going to continue to be the case. It's going to be a platform ... And by the way constraint of expression you know can have its pros and cons. I mean, we could all probably imagine things we would not want out there you know. Most people can at any rate.
And so it's a complicated issue how society should decide what is beyond the bounds of acceptability. There have always been some rules and I'm old enough to remember a time when the gatekeepers were very strong and entrenched in the establishment and there were three TV networks in the United States. And you couldn't on the Dick Van Dyke Show ... even though Rob and Laura Petrie were married they weren't allowed to sit on the same bed fully clothed. Which look, I'm not going to judge that particular call but the point is, the constraints were very narrow and there are still constraints on the big platforms. If you don't have one of the several big social media platforms open to your content, you can't get it out there in a broad way. That's the constraint. And, I don't think AI per se is going to be the big creative constraint, personally.
Speaker uncertain / overlapping speakers (context): When it comes to constraints, I want to bring you in
Speaker B (context; identity uncertain): Because social media has been terrible at bringing that kind of autonomy to the rest of the world.
Speaker uncertain / overlapping speakers (context): And there are parts in Southeast Asia in particular where issues of the damage that can be caused when those kinds of platforms aren't managed properly ...
Speaker B (context; identity uncertain): So how do you reflect on this conversation?
Speaker E (context; identity uncertain): Yeah, I mean, that's certainly true. Look at the example of Myanmar and what has happened with the completely unrestrained, in the beginning platform of Facebook under Meta and the violence that eventually caused.
But I think I can see both sides of the argument here. And two things can be true at the same time which I think is the case with this conversation that we're having. And one way I look at it is if we look at the downstream effects of not just social media, but increasingly AI now it does offer that degree of independence. And the fact that we're even talking about cognitive sovereignty it says something about how much we've ceded in terms of our agency to these corporations and institutions. I think, Evgeny, his point is perfectly valid, though.
At the end of the day, we're still very dependent on only a few actors, primarily within the Global North. We're dependent on them for infrastructure for standards, technical and policy standards that determine what we can do, what types of chips we can use in order to create these much more localised language models, for example. So at the end of the day, I think so many of us in the world are ultimately beholden to just a few powerful players corporations, essentially that determine how much sovereignty, how much freedom we can exercise even with the little crumbs that they throw us in terms of what sort of programs we can create with AI, for example.
The nation state of Tuvalu is the first digital nation in the world because it is at the precipice of an existential crisis due to the climate emergency. And a lot of the people of Tuvalu have decided they're going to migrate their whole nation state onto the digital platform in order to preserve what's remaining of their culture and traditions so that future generations can understand who they are in terms of their identity. But ultimately, whether they succeed or not in creating a digital nation is dependent on what sort of infrastructure they can access. The kinds of submarine cables the kinds of chips to power the data centres and the kinds of training data they're able to collect themselves. And so this gets to the point that I think both James and Evgeny are right. We're just looking at it at different ends of the spectrum.
Mohammad Hassan: And do you think this technology can allow these nations sometimes very small nations to develop their own sense of autonomy their own sense of sovereignty not only over their future, but also over their past?
Speaker E (context; identity uncertain): Yes, absolutely. But then we get back to the basic question of who does all this infrastructure belong to and what sort of standards are they setting for the rest of us in order to access their technologies? We talk about open models. Bob, you mentioned that, right? When you look at the AI action plan that the White House put out earlier this year it talks a lot about open models. But at the same time, Washington is trying to market an American AI stack with American values. So they seem to be completely conflicting yet at the same time, these are kind of the options that we're left with. There are open models, but we have no access absolutely no transparency of their training data. So we have no idea of the biases. No accountability in that sense. And even though these models are supposed to be open for use they ultimately come embedded with biases and political ideologies that many of us may not even be aware of.
Speaker B (context; identity uncertain): Is there a way to liberate ourselves from these biases or at least be aware of them when we're using these systems?
Speaker uncertain / overlapping speakers (context): Well, I think in terms of national bias, there is.
Speaker F (context; identity uncertain): I think the open source or open weights models that China in particular is focusing on are not very far at all behind the proprietary models that the US is marketing. And if I'm a nation that doesn't want to be dominated by American decisions about what a large language model should be and what its training data should be. And I do think it's outrageous that they don't even tell us what the training data is that they don't even disclose what journals it did and didn't train on and so on. But if I'm a nation that wants to be liberated from that kind of bias I'm going to find these open source models very appealing because you can retrain them to whatever extent you want and actually change the so-called weights which is what determines how they work inside and in effect create a customised model.
Now, at the individual level, that's more challenging. I mean, most people don't have the wherewithal to customise an open source model to them. At the same time there will be people who want to do it for people. I mean bigger players that know how to customise open source models and may create models designed for particular groups of people, and so on.
But this is going to create a lot of very challenging issues to the extent that they are customised. You know, we talked about the question of gatekeepers and freedom of expression and whether there's sometimes a societal interest in constraining expression, in some ways. And there always has been informally, at least there are things you're not allowed to say. And I just want to say that the issues that arise in this context are going to acquire whole new dimensions when people are marketing AI models that, for example, appeal to adolescents in particular ways but parents start complaining that they're having some insidious effect on adolescence and they're going to want legislation or something. And that's just a different kind of thing from like what film you're watching, what book you're reading. It's more complicated because we don't actually understand entirely how these models operate. And they'll operate differently on different individuals. So it's not like just watching one movie or reading one book and saying, OK, let's pass judgement on it. The model will play out differently with different minds. And I don't have the answers myself
Speaker uncertain / overlapping speakers (context): But the debate is going to get very challenging.
Mohammad Hassan: Because that's an interesting issue that already is appearing with some language models where you're starting to see parents concerned or even tragically, in some instances worried about the kind of self-harm that their teenage children who might be more susceptible to influence, have been led to. James, what do you think of that? Are these teething issues or is this going to be built into how these systems work?
James Brusseau: Well, I think all of us, of course, agree that there are some nefarious effects in these tools as there are in all new tools.
I suppose one question I have for Robert is I wonder if it's better to work within the system or to try to work from outside it. So for example, we talked a little bit about how you could fine-tune a language model that was originally made to suit Silicon Valley, right? That's the moral and cultural compass it has that's the way it works. But with tuning, perhaps you could drag that model working let's say, within the system of artificial intelligence within the language and logic of those people who designed the system. In other words, you speak the language of the people who live in Silicon Valley but you use their language, their tools, their ideas to redirect the technology so that it works for Java, let's say or Indonesia or some different place. So there's this idea that you could work within the system to achieve many of these ends that we talk about.
Whereas by contrast, of course, you could also work outside the system and that might mean something like legislation or rules imposed on some one or another level. I myself believe that the technology is moving so rapidly that working outside the system is not practical. And so, in this way also, I'm post institutionalist. I believe that even if we wanted institutions to be powerful and to be helpful, they could not be.
Let me give you just a quick example. So I live half my life in New York City and the other half of my life in Trento, Italy. I teach AI ethics in both those two places but because I'm in Italy, I've seen quite a bit of what the Europeans are doing with respect to laws. And one very important law for them was this AI Act which was this 18-month or two-year work to construct this regulating structure that was going to control AI finally and at last. And just as they were rolling it out, what happened? ChatGPT appeared. So we have this institutional rule it's about to be imposed, rolled out and rolled right back. Because ChatGPT rendered the whole thing irrelevant.
So it seems to me that the idea of institutions working outside the system will simply not work in a context of this velocity. Instead, these kinds of problems that we're facing can be managed, or perhaps we can only manage them by working inside the system, as I say using the tools of Silicon Valley. But let me use a word that's a little bit ... incautious, perverting the system. You pervert the system as it was constructed to turn it to a different end. So I think that's possible.
Speaker C (context; identity uncertain): - I disagree with everything in this statement in that this is a false dichotomy that you're drawing first of all, between regulation on the one hand, and some kind of innovation on the other.
First of all, the US regulates heavily. Look at all sorts of rules the US imposes on the rest of the world when it comes to exports of AI. The US regulates more heavily than anyone else what actually the rest of the world is allowed to do with technology. Who has access to chips. Who has access to what kinds of chips. The US is not by any means a laissez faire player.
Second, Europe is not a good example to follow in part because Europe's own hands have been tied by its dedication to some kind of neoliberal ideology which prevented it from building institutions that could balance and counterbalance actually what the market does beyond just regulation, beyond just creating laws. You can be creating massive public infrastructures that would actually be competing with the tech providers that are private and that come from Silicon Valley. Europe has not done that because it has been structurally prohibited by its own laws from building up its own tech industry. You're not allowed to do state aid in Europe and unfortunately, that's how it is. China, on the other hand, has been regulating heavily and combined it with active nurturing of its domestic AI industry. And we see the results. It's the only country that actually has something to show against the United States. It has a tech industry that actually competes with it and it's doing pretty well pursuing the regulatory path that we are told cannot actually work.
But beyond that, I would also be very cautious about thinking that some kind of technocratic, fine-tuning utopia where we just tinker with the weights of the model is going to save us, in part because much still boils down to the training data. And the training data is a very expensive project to undertake. And even if you play with the Chinese models you will see that despite the fact that they are open source and there are open weights what ends up being is that they still have the same ideological biases that the American models because they're trained on the same set of pirated books that the Chinese companies got off the same torrent sites that the American companies. And if you want to have a counter project you really need to undertake a massive government funded effort of going and finding alternative data sets.
It might be data in your libraries. It might be that you will need to go and digitise millions of books in your own language as opposed to relying on the English language books that are travelling around the internet. But you will need to have some kind of an effort to do that. And by the way, even the market players who have trained their models with that data from torrent websites they essentially leveraged the volunteering spirit of people who have been scanning books digitising them, and uploading them online. They did not incur those costs on their own. But to think that we will be able to solve that through some kind of technical fine-tuning of the model I think is very unrealistic. And in that sense, we do need to think about institutions and not just regulations and laws. Regulations and laws can be very useful if they're tied to an institutional vision and not just to regulating existing institutions which in this case are companies. Without inventing these alternative institutions we are not going to get very far. In China to some extent, invented some of them but did not invent all of them, and it's still halfway through in contesting the hegemony of the US in this space. Elina, what do you think?
Speaker E (context; identity uncertain): So, James, I would just offer that so many of us in the global majority work within constraints and we have no choice but to fine-tune because of the sheer cost of building your own foundational models that Evgeny has been talking about so much. And I think these smaller models, they offer some degree of autonomy but going back to the earlier point about how we would still be dependent on these few large players there's a completely unequal market concentration of power in terms of who gets what and so even fundamental concepts like time and space, for example that are embedded in the training data of foundational models we have no eyes on because there's zero transparency.
So for indigenous communities, for example that think of time and space as completely inseparable concepts for them to rely on foundational models with completely unclear, untransparent training data is going to change how they even fine-tune these models for themselves.
And that's not to mention the process of tokenisation and how expensive and arbitrarily expensive some of these processes can be. The tokenisation of the Vietnamese word for “a” is so much more expensive than it is for the English language. And again, that's determined by the designers of the model. Again, we have no eyes on this process.
And so for many of us, we have no choice but to work within these constraints. And so I'm sympathetic to Evgeny's argument that perhaps we just need to flip the tables and recreate institutions that work for the majority of us.
Mohammad Hassan: Is AI then a solution to that in your eyes, James? Can we train AI to then liberate itself from these constraints?
James Brusseau: Right, so one of the projects I'm working on is called curiosity or the curiosity engine. And so what we were trying to do with a number of students and myself is we are trying to work with recommenders. So again, what kind of jobs you're recommended what kind of language you might be recommended and so on. We're trying to work with recommenders to see how we can rebuild them or pervert them, to use my earlier verb pervert them to instead of leading back to our own data that's the individual level, or on the social level instead of leading back to the Silicon Valley data set. Maybe we could just call it what you're talking about the Silicon Valley data set, right? That's the underlying model and so on. So we're trying to find a way that recommendations which is one important aspect of AI can lead away from again, the individual's personal data set and interest. And I think that kind of project could be mimicked on the larger social level find a way to work again within the system to lead away from these Silicon Valley data sets. I think that's the challenge.
So the question is, How do we address that challenge best? I suppose here we just come down to personal preference. I'm fundamentally an individualist and so I'm going to be prejudiced towards saying that there's an individual path to these solutions. But at the end of the day it's going to be very difficult for me to defend that on theory or even on empirical evidence because we all need to start somewhere. And one of the places that I start is with the power of the individual even faced with these kinds of massively powerful machines.
Speaker F (context; identity uncertain): Yeah, I certainly agree with you that we should only resort to legislation when that's necessary. And a lot of problems you can solve without that. I would like to point to a category of problems that I think are going to call for policy solutions and they're going to call for policy solutions at the international level. And I think this ultimately gets back to the challenge of the technology being kind of tribalising at the national level.
I mean, if you look at something like the threat of biological weapons there's no doubt that these models are going to make it easier for people to do various nefarious things individuals and small groups of people make biological weapons and so on. That problem is particularly acute actually for open source models and it's why some people think that they shouldn't be developed. But in any event, that's not a problem you can solve through national policy. I think we'd all agree that preventing a global epidemic caused by a bioweapon is a valid policy goal but it's not a policy goal you can address at the national level.
And I think this is just one example of many policy challenges that AI poses that require a concerted international solution. And very broadly, I think if we don't confront the challenge of AI as a cohesive global community, we are in deep trouble. And so if it turns out that the technology itself empowers nationalism, that could be a kind of paradoxical and very deep problem.
And there's one other nationalising problem it's creating and it's that Silicon Valley ... Evgeny rightly pointed to the fact that Silicon Valley sometimes uses subsidies of various things to promulgate its values and its policy goals and so on. And right now, much of Silicon Valley is wed to this idea that there's this existential race between the US and China to get to superintelligence first. I think this is the absolute worst environment in which to confront the AI revolution. This is like a path to planetary suicide. And, the dynamic is very strong. The impetus behind it is very strong. I mean, the ... ten, twenty years ago kind of China hawkism was not the kind of de facto ideology in Washington. It's become that for more than one reason. But lately, one big reason has been that so many big Silicon Valley players like OpenAI want Americans to be as afraid as possible of China so that they can say that the American government should not regulate OpenAI or put any constraints on it whatsoever and indeed should subsidise its energy and whatever else. So ... I think if indeed AI has these kinds of tribalising effects at the national level and really at other levels that will make it harder to address the challenges that AI itself poses.
Speaker B (context; identity uncertain): And it seems that we're still in the very beginning of whatever this technological race could be between America and China. Other players are definitely going to emerge. Evgeny, you spoke about the EU and its attempts to regulate but then also some of the drawbacks to its own ability to do so. What happens then to us as ordinary users of AI, ordinary citizens? Do we then get shoved into these different categories these battle lines that are being drawn up?
Well, again, I think that AI, this whole debate
Speaker uncertain / overlapping speakers (context): It's not about the technology, it's about how we are going to live across many dimensions in the future. Those dimensions have to do with culture, with education with economy, with work, with politics and so forth. There is no dimension, I think, of modern existence that is not in one way or another it's not touched upon by this problematic, if you will of artificial intelligence. But we have to understand where we are. We are, if we were to draw some comparisons to previous inventions think about AI or at least large language models which is a big innovation compared to what came before we can compare it to the invention of the engine that powered cars and other forms of transportation.
Speaker C (context; identity uncertain): But there are many different ways in which that technology travelled from the invention. One set of consequences gave us Uber. Another set of consequences gave us public transportation. And it's the same technology driving it. And of course, if you were to leave that decision and that privilege to define the social imaginary, if you will, to [American industrialist] Henry Ford we would have never arrived to the public transportation option. We would have always been thinking about transportation as a commodity that should be bought and sold by individuals with the collective option nowhere on the horizon. The same with large language models. I mean, right now, if I were to delegate the question of How do we reinvent education, culture and work to [CEO of OpenAI] Sam Altman, OpenAI, Gemini and Anthropic? They will come up with answers that creativity is feeding a prompt into a machine and getting a video come out of that.
So your entire creative act is typing a creative prompt. Well, is it all that large language models can do to unleash our creativity? My answer is no. Then you need to have other institutions who can tap into the infrastructure, the basic infrastructure of large language models, to deliver on that vision to build these intermediary steps that will cultivate those skills in you. This is traditionally what institutions of education and craftsmanship have used to do. But that requires money, investment and an alternative political vision. If there is no effort to cultivate, build and promote that vision we are going to define and default to the basic path and that will be the path that the corporations would prefer to themselves that would align perfectly with their business models which will be selling us on the idea that education and creativity it's not about cultivating skills and individuals but it's getting stuff done as efficiently as possible which is kind of what their vision is with chat based AI.
Everything else requires hard work and it requires institutions and it requires being able to see through the politics of this and articulate an alternative vision. And that requires leadership, which unfortunately most of our existing political institutions are not ready to show. It doesn't mean that institutions are gone. It means that for various reasons, they are operating under assumptions that they themselves should be questioning. Should we be delegating these decisions just to the market forces as is the case when we delegate them to tech companies? Should we be empowering other players, trade unions, for example? Why aren't we listening to them? In Europe and the rest of the world, they're still a big force. Why aren't they a big player in this debate? These are all the kind of questions that we need to be posing because the nature and the content of AI as a technology, as opposed to as a social system as opposed to just the infrastructure ... If you go back to this comparison of the engine leading to both Uber and public transportation which one of the two is going to lead? It's still an open question. And we should not default into the answer that Silicon Valley imposes on us.
Speaker B (context; identity uncertain): Elina, what do you think of this idea? Is there a way to imbue AI with the right set of values? Is there even a right set of values for us to decide on?
Speaker E (context; identity uncertain): Evgeny brought up the point about leadership. I think more importantly, it requires time. And if you remember, I think it was about two years ago all these tech leaders called for a pause on AI development for six months. That has since gone away. And I think that's what we need right now this pause to really rethink what it means to be living in the age of AI where we're in some sort of race and the whole world is caught up in this competitive race between the United States and China. So we have zero time to sit back and reflect on what do we want AI to do for us. And this is where your question about values comes in. If we want AI to do X, Y and Z for us then what sort of values are we going to imbue it with? And for most of us, we just haven't had that time or space to really think about this more fundamental question. Instead, we're all on some hamster wheel racing to who knows where.
Speaker F (context; identity uncertain): Can I just reinforce that point quickly? A common thing you hear when you talk about putting any constraint on AI at all especially at the policy level, is, Oh, well, that will stifle innovation. I think with this technology, given how powerful it is how many new challenges it's creating and how fast it's moving stifling innovation is a feature, not a bug. Slower would be better.
There are a lot of things that may work out fine in the long run. For example, job displacement. Some people think that ultimately the people who lose their jobs will find new jobs. I'm not sure that's true, but even if it is there is still such a thing as dislocating pace of transformation. In other words, even if in the long run these people find jobs if a lot of people are hit at once with unemployment that's a socially destabilising thing. And I could point to a lot of other areas where maybe we can handle the challenge maybe we can adapt to the technology but we're not going to be able to adapt across as many fronts as we need to avoid national and even global destabilisation if the technology moves as fast as the people in Silicon Valley want it to move.
Mohammad Hassan: Now, James, you know I'm coming to you for this.
- I'm sure he has a response. - Because on the other end of the spectrum, there is what is called the Accelerationist movement, or Effective accelerationism which is how you define your own position. Could you explain to us what that position actually means and why you stand where you do?
James Brusseau: Sure. Actually, I'm not an Effective accelerationist. I'm an Ethical accelerationist which in this very, very small world is a very considerable difference. And we'll come back to that. Small world of 17 people or so.
We'll come back to it. I'd like to maybe intervene just quickly on what we heard from the last three thoughts. It seems to me, on the question of innovation and what people can do in the face of these larger institutional structures which of course exist, that's undeniable if we look at what's happening on the ground the story of what reality shows is not that these institutions are dominating, just the opposite. So, for example, one of the projects I'm working on is called the Caffeinated Professor. What is the Caffeinated Professor? He's a professor who never sleeps. – That's the reality. – That's the real thing.
But the idea is, here's a problem the students have. Students are frequently ashamed or afraid to ask questions of the professor. They're afraid that they will look unprepared or worse because their question is not very perceptive. So one way that we are devising to help students in this situation is we're creating a digital twin of me which is a large language model trained on the lectures which have been recorded for many of my courses. So we're going to have a me which comes close ... an AI me which comes close to being able to answer questions not as a general AI, but as James Brusseau the individual would answer that question.
And the idea is that if we can make this tool available to any student at any time without even making it available anonymously they don't need to sign themselves in then they can do the following things. Of course, Caffeinated Professor, they can ask a question at 3 a.m. when I'm happily sleeping. But perhaps more importantly, they can experiment with questions that they're not certain of. Or perhaps sometimes students have a question but they don't know what the question is. And so they need to go back and forth with the professor a little bit to try to find what their uncertainty is.
Now, for me as a physical person, I just can't do that. I have classes of 40, larger, 60, 70 students. I don't have time to go over with every student a long discussion about what their uncertainty is. But my digital twin could. So what's happened in the world, it seems to me is that individuals are in fact proposing solutions creating opportunities which run in favour of the kinds of things that all of us agree we want and which don't necessarily go against Sam Altman and OpenAI but certainly do not dovetail or exclusively dovetail into these economic kind of concerns. So I think that just generally what's happening in the world disputes a little bit this fear ...
Speaker C (context; identity uncertain): - What is the moral of this story? What disputes in that story? Exactly what? Sorry, I don't understand the nature of the argument. What exactly that story is supposed to dispute?
James Brusseau: What I'm trying to say is I believe that this tool can help further student education and in that furtherance exists outside of ...
Speaker C (context; identity uncertain): But nobody's disputing that. Nobody's disputing that you can take an Uber somewhere and it's a great tool. But I also like public transportation. So what does it dispute exactly?
James Brusseau: Well, it's disputing the idea that education, when it's using AI is being funnelled into this large Silicon Valley technocracy.
Multiple speakers (context; identities uncertain): - No. This is a strong argument ....
Speaker F (context; identity uncertain): - But I think it's true that often people who want to see some constraints placed on the technology tend to belittle its potential value. I'm sure you've run into that and that's kind of what you're responding to.
Speaker uncertain / overlapping speakers (context): None of us are making that case, though.
Multiple speakers (context; identities uncertain): - What's that? - None of us are making that case.
Speaker F (context; identity uncertain): That's true, but it's very common. It's almost a strange correlation between your estimation of the power of the technology and the potential benefit and how you think it should be handled. My own view is it's very powerful full of potential benefits. I mean, you know already my dream which is to be able to have a personal conversation with the world's leading authority on anything is we're pretty close to and it's an amazing experience to be able to learn interactively with that.
Similarly, I'm sure this is going to bring to much of the world that doesn't have access to great medical care the ability to get a better diagnosis than is available to them right now. There are tons of potential benefits.
I don't deny those and you probably don't either. But ... I think the very power of the technology is the reason we have to think carefully about how it needs to be channelled. And again, I just don't think ... You know, Silicon Valley has been very effective in discouraging any meaningful regulation. I mean, it seems to me like the most obviously legitimate request, for example, to say You're required to at least tell us what text it trained on. Just tell us what the training data was. What would be the problem with that kind of transparency? I'm curious as to whether you'd be opposed to that. I kind of suspect you wouldn't have a big objection but even that seems to be too much to ask. Nobody's even talking about it in America.
Speaker C (context; identity uncertain): But that's a great example, though, because I think it also clarifies the differences maybe between us because in my case, my argument is that because of the fact that this basic infrastructure of large language models has been usurped and kind of appropriated by Silicon Valley, we're actually under-innovating, because other institutions cannot actually have meaningful innovation in this field without having a lot of resources coming from the public sector. Right now, if you are a government in Europe and you have to decide whether to give money for experiments with AI to a public university a library, a trade union or to Sam Altman to build another data centre you're going to give money to Sam Altman, right? And that's unfortunately what the reality is. And this has been the case with Silicon Valley throughout the last 20 years or 25 years.
I look at many of the tools. I look at Google Scholar that Google has built. It's a tool with wonderful potential but it's performing probably at 10 percent of what it is capable of. You give it to any public institution like CERN in Switzerland you're going to end up with a much better tool and a much better infrastructure. The same case would go for AI. I'm all for giving faculty, professors and students more tools to play with. I just don't think that Silicon Valley and OpenAI and others are interested in meaningfully giving us those tools. We can have some by-products of the normal commercial activity and we can kind of hack our way through them and innovate and maybe even use and deploy them.
Mohammad Hassan: But this is not a meaningful long term strategy when it comes to education, culture, employment or politics. So, James, I want you to define for us what accelerationism is and how it responds to this question of regulation or slowing down.
James Brusseau: Sure. So ... Basically, as I understand it, there are sort of two camps. There's the accelerationist camp and then the precautionary camp. And each of those camps splinters into various sub factions that are warring and unhappy with each other. But generally, it seems to me that it's right to say that the accelerationist belief is that, in general, innovation itself will resolve innovation problems.
So, for example, one innovation problem that AI has obviously caused, as professors know is that students are cheating right and left. They're just doing their homework with the machine and turning that in letting AI do their exams and so on. So a precautionary approach there ... Or rather, let's start with the acceleration. The acceleration approach would be to say, OK, we have this problem. AI has caused that. Let's use AI itself to resolve this problem. How might we do that? One way we could do that is by developing an AI professor sort of like the kind I was discussing and having this professor administer oral exams to students. So literally, the same technology which allows students to cheat, then breaks that path and might provide better kinds of exams for students than we have had in the past. In the past, students have been writing in blue books or things like that. Oral exams are much more effective but it's impossible for a professor to oral exam a class of 40 students. Not enough time. Perhaps with the digital twin that will become possible. And we can grade students in terms of how well they're able to respond to the conversation the AI leads or how close their responses are embedded in two dimensional space of language, let's say how close their responses are to what the AI would have produced for one or another question.
So acceleration is that idea. It is that AI will cause problems and these problems should not be solved, let's say, through institutions not be solved through ...
Speaker uncertain / overlapping speakers (context): I'm not sure exactly what you're proposing but ...
Multiple speakers (context; identities uncertain): - A slowdown. - A slowdown, right? - Not result through a slowdown. - A pause, yeah. - Right, that's what I meant. A pause. - Yeah.
James Brusseau: But instead should be solved by accelerating, by going faster.
Now, I will first let you speak for what the pause is. But I will just say quickly the way we accelerationists understand what the pause means and I defer it, of course, to your definition but the way we see it is, the go slow group will say Let's halt innovation until we can be certain that we have under control, fundamentally the kinds of risks that any particular technology will create. Our practical response to that is that we cannot foresee those risks until those arise. So for that reason, we make the practical argument that acceleration is a better path to follow than what I call the precautionary approach or what you might call the pause approach.
Mohammad Hassan: I'm going to let you respond to that, Elina.
Speaker E (context; identity uncertain): Right. So I fear that we're entering into very dangerous territory if we say, Well, we don't know what the risks are therefore, we're going to keep innovating until we arrive at those risks and then we'll deal with those later. And those risks can affect different people very differently. Like Mohammad, you brought up the example of how social media impacted places like Myanmar, you know, very violent and there were deaths involved.
And so at what point do we say Well, what types of risks are acceptable in order for us to pause or slow down on this innovation? And I think this word innovation also deserves a bit of unpacking because innovation may not mean the same thing to me as it does to you. And so we need to question this premise that innovation is necessarily a good thing for all, equally. And I don't think the experience on the ground has shown that to be the case.
Bob, your question about why don't we have access to this training data. It's because it's the secret sauce of all these companies. They're not going to give up their proprietary hold on this data in order to make money. But it does, I think exacerbate this idea of this hierarchy of knowledge. And so this question of innovating particular tools for different use cases like education, healthcare they may mean something very noble at the surface but I think if we dig deeper, it assumes certain things that there is data that is written that is corroborated, that is verified and it assumes this knowledge, that written knowledge is superior than oral traditions, for example. And there are many communities in the global majority that rely on oral traditions in order to pass down knowledge but in order for them to fully reap the benefits, as you say of this accelerationist movement in AI then that knowledge will have to be documented and a lot of communities simply A, don't have the resources B, don't have the time and C, may not even be inclined to think of knowledge as purely just written. And so we ran into different levels of assuming a superiority of knowledge but also this flattening of norms and cultures and histories in order for a lot of communities to be a part of this AI game because they feel they're missing out if they don't participate.
Mohammad Hassan: And then what do you say to James's argument that if you stop, then you don't get to innovate in a way that could potentially bring about the solutions to the problems that you're seeing?
Speaker E (context; identity uncertain): But what are we innovating for? And I think if you talk to the technologists themselves they'll say, Well a lot of it depends on the use cases and what sort of tools are being innovative for particular communities and for particular use cases may not always … We may not necessarily have to rely on AI to innovate. Why can't we think about other ways to innovate solutions? And this gets back to Evgeny's point. Perhaps we're under innovating in that sense because we're over-relying on AI.
Speaker F (context; identity uncertain): And can I just say that there is something in between a pause and accelerationism which is just don't discourage the things that are worth doing and would slow down innovation. For example, suppose you wanted to put a tax on data centres on the grounds that they are contributing to climate change because in the near term it is going to increase carbon emissions. The AI companies in America would say, no, we have to beat China and that would be the end of the conversation. And in the current political context in America, and by the way, I'd say this is another case where there's an argument for international policy because the other thing you can say about something like that is wait a second, no other countries are having to make this sacrifice. Why should we? And so on.
So first of all, it isn't just pause versus acceleration but also I wouldn't exaggerate the extent to which the kinds of pauses that are being proposed would actually stop or even radically slow innovation because the pauses are about training whole new large language models. The next generation. And I think most people in the field would tell you that if you didn't train any new models it would take a couple of years for the innovations based on the current platforms, based on the current models to play out and for people by adopting them and giving feedback about what they want to lead to the creation of new products.
And the other thing I'd say is that although I think there are a lot of cases where AI will help solve the problems it creates, eventually … First of all, maybe not always soon enough. For example, if there's a whole new kind of cyber threat posed by autonomous self-replicating AI it could be that the consequences of that are so grave that it doesn't help that two months later you have a counter AI that can handle it, right? If, meanwhile, you've taken out all the satellites and they're not functioning or whatever.
And then there are kinds of problems that I'm not sure AI is going to help us with much at all. Let's suppose that job displacement is permanent as a number of people think it may be because once you've automated the brain it's like there's no part of the body left to automate. I mean, we've already automated a lot of manual labour and people think that with robotics, we'll be able to automate the rest and we'll be able to automate all office jobs and so on. If it turns out that unemployment is permanent I'm not sure AI is going to give us a solution. I mean, it's going to be a classic policy problem to some extent. Like do the rich people want to create universal basic income or whatever that'll give everyone enough to survive on? And also how will people spend their time? Will it be productively or what? I mean, I think AI will help to some extent with some of those questions but I wouldn't expect it to give us a miracle cure in the near term. And that's one reason I say slower is better.
James Brusseau: I'm not sure. I don't think any of us are certain about whether in the real world slower or faster will turn out better. I think that that's fundamentally unknown or unknowable, perhaps even.
And I think that that comes back to a question you asked Well why do we innovate? And one of the aspects of the ethics of acceleration which is different than Effective acceleration is the idea that fundamentally for those of us who promote this view fundamentally, what is unknown is good. We want what is unknown. We want to advance into it.
So that would be sort of my response to that. So how do I explain this value or these ethics? It's something like the following. I think that all of us have travelled a long way to be here. And I think most of us have seen that there are two kinds of travellers, right? There's the traveller who before they leave planned which hotel they will stay at which restaurants they will eat at which places they will visit on each day. And then they follow this itinerary and go home. But then there's a different kind of traveller. There's the person who buys the one-way ticket and says, I will arrive at the airport and I will find a hotel. Then I will find a restaurant. Then I will find something to do. And if you ask, well, Why does that person go? What is that person looking for? I think that person goes because they're attracted not to something that they might find. They're attracted to the fact that they don't know what they're going to find. And there's something in the unknown itself which is … which acts as a kind of magnetism which we want and which I believe is profoundly human.
And so going back to the question, Why do we innovate or, Why do we enter acceleration? I think fundamentally, that's the reason. There's a human reason underneath this movement of technology. It's because we want what we don't know at least those of us who buy one-way aeroplane tickets because obviously there's other human beings who disagree with me just on the straight human level and say, What are you talking about? You've got to get an itinerary. You've got to know the hotel you're going to stay at. We all know people like that, right? Where are you going to eat and that kind of thing. So I think we come down to sort of a fundamental human difference here, which is going to shape the way we think about that.
Speaker F (context; identity uncertain): I would say about that metaphor is with this particular technology there is going to be plenty of unknown and unanticipated even if you imagine that governments do as much as is realistically expectable to constrain it. There's a tremendous amount of impetus behind the development of the technology. And every week you're reading in The New York Times about some new weird thing that people are going to have to grapple with. I just think there's going to be plenty of unknown stuff being revealed. A lot of it's going to be great. There will be scientific breakthroughs. I do believe, I just worry that there's potential for true catastrophe if we're not careful. There's potential for bad trips, right?
Multiple speakers (context; identities uncertain): - We'll buy a one way ticket. - Well, right. - And wish we'd go back home. - A bad trip is just a bad trip.
Speaker uncertain / overlapping speakers (context): The apocalypse is the apocalypse, right? Like, you don't even get frequent flyer miles
Multiple speakers (context; identities uncertain): - With the apocalypse, so … - yeah, yeah.
Mohammad Hassan: Evgeny, what do you have to say about this analogy of these two travellers, these two ways of approaching what this technology could bring to us as human beings?
Speaker C (context; identity uncertain): Yeah. My general view on dichotomies is that they are often fake and problematic. So in that sense my comment, but also question to James is as follows. So my hypothesis about accelerationism is that it's a form of religion. And as all religions, it has a blind spot but it's not a religion about technology and the blind spot also it's not about technology. The real blind spot and the content of the religion is really about capitalism and the market being the only driver of innovation. And ultimately everything else flows from there because there is no discussion what kind of comparatively different setups could give us fast innovation, better innovation qualitatively different innovation? All of these discussions about alternative institutions alternative setups, alternative property regimes alternative access regimes all of them are sidelined and we are focusing only without saying it explicitly on whether the current corporate dominated model is going to give us better results than regulation. And for me, that blind spot that we are not even considering for a second an alternative setup, that, to me is a hallmark of a religious doctrinaire ideology.
Let me give you an example. If you now want to experiment with AI, you have two options. You can be doing it as an individual, user and subscriber too many of these services. But, again, to experiment with the most advanced models you'll have to pay several hundreds of dollars a month just to one company. Or if you are really advanced and you really want to get something advanced out of them you can be tapping into their so-called Application Programming Interface [APIs] into their more advanced offerings that normally are reserved for companies government institutions and anybody who can afford to spend tens of thousands of dollars a day or months on paying those fees. Obviously, those institutions can innovate and they do innovate a lot but there are plenty of actors in society who, if given access to that same basic infrastructure would perhaps be innovating differently.
Perhaps they would be innovating faster. Perhaps they would be solving problems that the corporations are not solving. And the current corporate dominated regime in which we offer AI does not favour the presence of these players in the innovation sandbox. So if you really want to be ethical and consistent with the principles of Ethical accelerationism the real agenda, in my view should actually be encouraging completely radically redefining the institutional landscape on which the current AI works and really making sure that all of us have the ability to innovate in the same way that an entrepreneur with access to venture capital can because otherwise we are creating a society where without dreaming of becoming a start-up you have no way to innovate with those AI. You can only innovate as an individual consumer paying hundreds of dollars for individual chat-based interface. And everybody who has worked in both worlds both with the API and individual access will know that the real innovation happens elsewhere. It happens with the companies and governments are innovating paying hundreds of thousands of dollars a month and not where the individual users paying hundreds are innovating. That world does not exist for anybody but companies and start-ups.
Mohammad Hassan: - James, what do you think? -
James Brusseau: I guess we'll see. I mean, I've proposed several projects I'm working on with real students and real universities. We'll see what kind of success we have.
Speaker uncertain / overlapping speakers (context): Yeah, but this is an imminent critique of your ideology.
Speaker C (context; identity uncertain): You have to operate within its terms. I have just operated with its terms. You set out what counts. What counts is novelty. And I've shown you that your own ideology if you follow its premises actually blocks the production of novelty. So you have to respond to that. You can't just say, we'll see if you preach a coherent ideology you have to be able to defend it.
James Brusseau: Well, the ideology I was promoting here is ... separates this brings us back to the Effective acceleration versus ethics acceleration. And you perceptively noted that there are some who have this almost religious affection for the techno capitalist machine. That is the Effective acceleration crowd. The Ethical acceleration crowd, that other side where I am that's exactly the point upon which we separate from them. And the way we separate from them is by saying that our thoughts about artificial intelligence should fundamentally be based upon individual human experiences. And so in our talk today, I've tried to, at every opportunity I've tried to talk about things that are actually happening with real people in the real world. And it seems to me that it's on that level where I can maintain a strong case.
I can say, Look, we are using this tool to create better outcomes for students. I don't need to be Sam Altman. I pay 20 bucks a month. I don't pay this $200 a month. And with the students I'm working with we are accomplishing and making progress in creating this chatbot that I've discussed.
Similarly in the project with the curiosity engine that is trying to break from within this force, that the AI recommendation system has to reproduce our past and get outside that. We are having some luck with something called orthogonal recommendations, which we will not go into. But these are kinds of things that individuals at universities can do and are perhaps having some success.
So I think that I would respond to your forceful argument in the following way. I would say, first, there is a difference between Effective acceleration and the Ethical acceleration and I'm on the ethical side. And second, I think that my citation of actual practices, things I'm doing things that students are working with me in doing. Those are forceful theoretical arguments, too.
Speaker C (context; identity uncertain): If we can make it work.
Multiple speakers (context; identities uncertain): - If we can't … -
Speaker C (context; identity uncertain): Just because you can raise funds to pursue interesting projects at the university does not make certain parts of reality disappear. And the fact that they become invisible to you does not make your project more ethical. It makes it less so, if I may.
Speaker E (context; identity uncertain): All right. Can I jump in here? So I think what we're seeing, James, as you've described is happening quite a bit on the ground in Southeast Asia itself. And so you have a lot of local innovation happening for particular use cases, as in education as in the financial sector even in the criminal justice sector, which is slightly disturbing but we won't go into that right now. I think, though, that there are some bright sparks of innovation.
And just for clarity, I'm not arguing against innovation. I think the importance here is the nature of innovation. What are we innovating for and what are the opportunity costs that we should be considering perhaps more effectively?
Now, there are bright sparks in the innovation in terms of the governance models around AI that we're seeing, particularly with the indigenous communities where they are challenging some ideas that are out there about what it means to govern AI in a space that prioritises exploitation and extraction particularly of data, but also other natural resources. And the Maori community in particular, have decided that they're going to have their own governance models of their own large language models because of their history of having been exploited. And I think there are lessons to be drawn here for many of us including at the local levels, as you've tried to argue, James that even at the policy level we can see some ethical turns of innovation that might not change the fundamental structures of the AI reality right now but that may begin to chip away at some of the status quo institutions.
Mohammad Hassan: When we think about these indigenous models that we're seeing growing does that then present a future in your mind of the potential for AI to allow these different communities to create their own tribes their own spaces, that they then can express that autonomy that sense of sovereignty?
Speaker E (context; identity uncertain): Yes. And there are certainly some really positive outcomes that will result from that sort of innovation. At the same time, for many multi-ethnic communities and nation states it does present a bit of a risk because it means a fragmentation of the very social fabric of a nation. And I'm thinking here of a lot of countries in Southeast Asia but also in Africa where, for example, there are local language models that are being built in terms of ethnic languages. But the history of these countries, as in Indonesia, as in Malaysia has been built around unifying people through a single language. And so when you have this splintering of innovation, of models that are meant to be ethical in terms of how they cater to local communities there is the larger question of what does that mean for national unity and how do we bring those contradicting tensions together?
Mohammad Hassan: This is a point about nationalism, which you brought up before, Robert. How do you see this happening? There's obviously a driver now for nationalism and nationalistic pride especially when it comes to things such as the defence industry which we're seeing a lot of companies in Silicon Valley voluntarily stepping into. And there's this race now between the US and China. Do you think that we can be getting these models that are reinforcing these identities? And is that a good thing or a bad thing?
Speaker F (context; identity uncertain): Yeah. I mean, first of all, the Silicon Valley joining of the military industrial complex is a whole issue of its own. This is a recent thing. We have a whole new ton of money devoted to supporting militarism in the US.
Now, as for the question of nationalism and I think you're also bringing up more broadly the question of just segmentation intra-national division along ethnicity or anything else. I think all of these kinds of fragmentation are real issues. AI has the potential and even in some ways, the tendency to create narrower and narrower groups of affinity. In some cases, that's fine if it's like Boston Red Sox fans or whatever. But of course, in some cases it's not. And take the United States, there is intranational polarisation which itself is an obstacle to coherent policy about AI and coherent policy about a lot of other things. And then there is also the division along national borders which can indeed be exacerbated by AI is being exacerbated by AI companies for their own political reasons. And again, I think nationalism is always dangerous and intra-national polarisation is always dangerous. But I think in a way it's going to become more dangerous than it's ever been by virtue of AI. I really think this technology is going to be so powerfully dislocating across a number of dimensions and in many different ways called for a concerted international response that if we become more of a global community than we've ever been before we're in really deep trouble.
Mohammad Hassan: James, do you have this concern that AI might entrench us in our already divided lines, whether it's national or race?
James Brusseau: Right. So I think it's becoming fairly clear from our discussion so far that I start almost all of my thought from the individual level. So I would say two things. I would say one, almost certainly that's true. We're seeing clearly how filter bubbles are forming. We're seeing clearly how polarisation increases. We're seeing clearly how people are being directed into narrower and narrower affinity groups. That's just undeniable on the ground, really. But again, going back to sort of where we started it seems to me that the answer to that question is one for AI companies to work with individuals to help them find new ways of thinking about the world which would be not moving from one national structure to another national structure or from one language to another indigenous language. Instead, I think it would mean for individuals to find their own way to kind of language that they might want.
For example, perhaps I might decide that I want to learn Italian. I want to live like the Italians. Now I have no Italian past. I have no affinity with the Italians. I have no indigenous connection with that. But why can't I become Italian as opposed to recreating sort of the roots I had with my … as it happens, my ancestry is Swedish and Danish. I could go back. There's things called lutefisk that we could eat and there's all kinds of specific things we could do but I could just go forward too, right? I could use artificial intelligence and tools that it offers to help me gain entrance to a different kind of culture. So I think in that way too this question about nationalities is shattering and instead it will be up for individuals to find their own place and their own destiny in this future.
Speaker F (context; identity uncertain): And I think one hope generically about the formation of new tribes that are not national tribes is that some of them will be first of all, benign not bad and international. They will cut across national bounds and so form commonalities of interest within multiple nations and so erode the power of nationalism.
Mohammad Hassan: I wonder what you think about this path forward this potential for division, this potential for autonomy and the potential and the reality of the acceleration at which this technology is going. Where do you see it going forward? And how do we protect ourselves while also not denying the kind of innovation that is possible?
Speaker C (context; identity uncertain): Yeah. Well, I, unlike James do not start from the individual analysis. So I look at the actually existing tech companies and not some hypothetical firms out there. The actually existing tech companies all go and their CEOs all go and genuflect in front of Donald Trump. They have funded half of his White House reconstruction. They are part and parcel of the nationalistic and you might even say, jingoistic agenda of the United States.
So the idea that I would want to have more of these companies intermediate my life to help me learn Italian. No thank you. I became an Italian citizen, learned Italian. I'm doing fine. I don’t need Sam Altman helping me along the way. By the way, all of that is true. So in that sense, no. I just don't know what universe do we live in? I mean, these are real actors.
Nvidia is soon worth, maybe already worth, $5 trillion. It's a company that's deeply embedded now in Trump's project for redefining the American hegemony and the American empire in the world. How can we possibly be making these hypothetical statements about what these companies might do to create a more global universe? They are all residents of the United States. We have seen that despite their early statements even during the first Trump administration they all ended up playing up to Donald Trump's tune.
I still remember in 2016 and 2017 when people like Sergey Brin were going to San Francisco airport to protest against the first ban on the Muslim countries. I mean, now Sergey Brin will probably be transporting people to the airport if Donald Trump asked him to. This is where we are. These are actually existing tech companies and they're the ones building and rolling out AI. Do I want to have this powerful set of infrastructures in their hands or do I want to redistribute control over them more widely? Of course it's the latter. Why would I be, not being an American citizen why would I want them to dominate it and control it and make it part of this somewhat insane Trumpian project?
Mohammad Hassan: Elina, I want to give you the final word here. When it comes to … You've talked about the need for caution the need to potentially stop but then there's also the realities of what is actually happening in this technology. And on some level, it's very difficult to actually put a stop to it. So from your perspective, how do we move forward from here? How do we protect ourselves from the worst risks that we already see while also unlocking its potential?
Speaker E (context; identity uncertain): Big question for us to end on. Let me try my best. So I think it's really important to have friction points at different levels. And again, I think it's really good to have innovation depending on what the particular use cases are and depending on the context in which these innovative solutions are deployed. That said, we've talked about kind of the communalism and nationalism that AI can engender. But I think to Evgeny's point as well there is a real risk of ideological tribalism as well. So on the one hand, yes, you democratise AI for communities that are underrepresented, languages that are underrepresented. At the same time, because of the structure of the economy that we operate in that enables this sort of AI innovation at large around the world we are inadvertently sucked into this ideological tribalisation of this sort of capitalism being the only way forward in order for us to "innovate." And so we are siloing some of the innovation that we should be exploring and confining ourselves to only a certain framework whether it's at the technical level, whether it's at the policy level whether the legal level.
And this is why I've tried to advocate for there to be friction points at different levels. At every stage of innovation we need to ask ourselves, what is this for and what are the opportunity costs that are incurred? We think of innovation, particularly in my part of the world as an economic driver but that also means taking into account conventional economic theory where things like the environment and labour are "externalities." And I don't think we can afford to fall back on some of these conventional ideas anymore precisely because the sorts of innovation that are being called for are going to require a greater lateral sort of thinking.
Mohammad Hassan: Very well put. And, before I lose you guys, I have to ask this. And I know it's a personal question but I would love to know what is your LLM of choice? Is there one that you go to? Is there an AI tribe that you feel like you already exist in?
Speaker uncertain / overlapping speakers (context): I was ChatGPT, but I'm a believer in Gemini 2.5 now.
Speaker uncertain (context): Robert?
Speaker uncertain / overlapping speakers (context): They change so fast.
Speaker F (context; identity uncertain): The innovations come so fast that I have trouble sticking with one favourite. I actually … Maybe I should be ashamed of this but I pay $20 a month to Google, and OpenAI and Anthropic.
Speaker uncertain / overlapping speakers (context): I'm sure they appreciate your money.
Speaker F (context; identity uncertain): It's the one thing they like about me. Probably nothing else. Elina, what about you?
Speaker E (context; identity uncertain): I refuse to participate in the subscription as a service model. And so, true to my advocacy of context mattering I play around with different models and I pick and choose as to which solution is best delivered for my particular context.
Mohammad Hassan: And Evgeny, what about you?
Speaker C (context; identity uncertain): OK, so I have to make a confession. So here it will be a real AI Alcoholics Anonymous kind of setting. So I have three Max Claude accounts for which I'm paying probably 600 or $700 a month. I have a Gemini Ultra account and I have GPT Top account so it all ends up to about $1,000 a month.
Mohammad Hassan: Wow. You are the ideal AI user.
James Brusseau: Everything reverses in the last minutes.
Speaker C (context; identity uncertain): I spend most of my day inside those models. I am actually experimenting with APIs and looking at them very deeply. So that's what you have to do. So I know what the burden is. If you really want to innovate, somebody has to pay for that. It's not just flowing through there you can just go and get on the cheap and the difference is immense.
Mohammad Hassan: Evgeny, Elina, Robert and James, I want to thank you for this very lively debate. I've personally learnt a lot and I hope that you've joined us on this conversation. We would love to hear your thoughts, so please share them. My name is Mohamed Hassan and this has been the Doha Debates podcast. There you go guys.
If you enjoyed this conversation and want to see more subscribe to the Doha Debates YouTube channel so you don't miss out on any future episodes.