Ashish Rajan: [00:00:00] I'm curious if any of those would be exploited in the sense that it would start screwing up the data that people are trying to produce for what AI content is, and if that becomes like, Hey, we have 90% on our platform using AI. Yeah, I would question

Caleb Sima: And in reality, only 10% is made with ai.

Everyone

Ashish Rajan: else. That's right. Everyone's actually general content.

Caleb Sima: I agree with all three of those. Dude. This is like we're gonna lose our jobs.

Ashish Rajan: So Google recently published a report on the misuse of generative AI. This is the real life scenarios. About 200 use cases were looked into from the education sector, from the health sector, and the public sector as well.

All these reported incidents were triaged for what kind of misuse was it for generative AI. So the little secret, there was some cybersecurity prompt engineering stuff in there as well, but there were a lot of other use cases that you may not have considered. So in this episode of AI cybersecurity podcast, Caleb and I did what we do best.

We analyzed it as humans first and shared our opinion as CISOs who are reviewing a document that's been provided by Google, but also to make it more interesting. Later on, we use ChantGPT as well [00:01:00] as Athropic's Claude to know what would they think? What would the AI think of this document we share with them?

And let's just say the results were interesting. I would say there was definitely a clear winner, but I'll let you watch the episode or listen to the episode if you're on the audio platforms and make the best judgment for what you think is the clear winner in this case. If you are someone who's researching the Misuse of Generative AI in real life, especially in the public sector, education sector, as well as a broader categories that consumers look at.

This is definitely the report that you should look into. I'll leave a link for the report on the show notes as well as the description. But if you are listening or watching the episode of AI Cybersecurity Podcast for the second time or third time and have been finding it valuable, I would really appreciate.

If you are on YouTube or LinkedIn, definitely give us a follow, subscribe. But if you're listening to us on Apple or Spotify, please give us a review or rating. It definitely helps more people find out about the awesome work we're doing here. So we get to know and share the knowledge with more CISOs and cybersecurity leaders who are [00:02:00] trying to learn about how real are the threats for AI, Gen AI and everything that comes with it and how to manage it.

Welcome to another episode of AI Cybersecurity Podcast. Today, we're talking about generative AI misuse, a document that was released by Google DeepMind, which is also a taxonomy of tactics and insights from real world data. As Caleb called it, these are real world attacks we saw. And we're just going to go through these documents and come out with insights for you guys.

But I want to call out first thing, which I read in the first paragraph. These are only incidents between 2023, January, 2023 and March 2024. And in between that they had approximately 200 observed incidents of misuse.

Caleb Sima: I think that what we see in this world today is a lot of FUD fear, uncertainty, and doubt.

And what they're going to come out with is this is what Google has seen in the real world. These are the things that we're witnessing. And that way it puts some data and some realistic expectations as to what's happening today. And so I think what I'd love [00:03:00] to be able to do is see based off of my finger in the air of people I've talked to, how does this apply? What are they going to talk about and how real is the document? But I love the fact that they're producing this so that we can get like real data.

Ashish Rajan: I do also want to call out that it's probably focused at everyone. So it may not have specific enterprise scenarios as well in there.

And we'll probably go through scenarios one by one in terms of the, what the findings were. I was definitely curious. I was quite excited, skimmed through it. Before we got on the call, I'm like, this is really good, but some of them look a lot more for companies that are dealing with day to day consumers and with the misuses, I think this is definitely a target for them.

I just want to start with the introduction. Cause there's something called out there. And I don't know how many people would know. It was the very first line in the introduction was generative multimodal artificial intelligence. And multimodal these days, I think from when we started doing, The recording for AI Cybersecurity Podcast. I don't think we ever covered multimodal because it only became a reality in the past few months when OpenAI made that [00:04:00] release for voice activated, hey, have it as an assistant. How would you define multimodal out of curiosity?

Caleb Sima: We didn't really cover, but we did say that it's coming and it's a no brainer.

I believe we talked about, even in our first episodes, how we laid out the way we communicate with LLMs today is via chat. The data that they use is public data. These there's this progression. And over time, the way you communicate will be via both chat and audio. And then the future will be chat, audio, video.

We covered multimodal in a sense where we talked about the progression. Yeah. And also think about it as. And the data that they use will be public and private data, right? Think about your personal email your personal calendar, enterprise data, all of those things will move forward. So we covered it that way.

But multimodal is very simplistic. Multimodal is your senses. So when you think about how you interact with the world, that's multimodal. Today, again, I'll make another prediction. [00:05:00] Today, we now talk to LLMs via chat, via photos, via audio. And then next as an obvious via video, you're going to interact with your LLM via avatars, physically look like people in terms of what you're doing.

And also, as we even move further, avatars and other in your LLMs will start responding to real time event data. So think about it as your GPS and location constantly being fed to an LLM. So it immediately says, Oh, Hey, Ashish, I noticed you're at home. Here's a couple of things I think I need to remind you about, or, Oh, Ashish, you're at home.

Let me turn on your home automation stuff for you. Think about it as, Oh, it responds immediately to photos and videos you're taking with your iPhone. So you're recording your kids and it will automatically say, Oh, I noticed you're recording your kids. Let me put together a nice little collage for you so that you've got it.

Oh, I also noticed because you're recording your kids in Hawaii, I'm going to send it to your parents. So they have a link to watch, like all of these. So think about real time data being [00:06:00] fed into it. These are like senses. And then as it moves into robotics, clearly, I think. You're going to turn on your microphone all the time.

Yeah. Think about how crazy the privacy people are thinking right now. If your audio is constantly running, it's going to be able to pick up conversations. It's going to be better to pick up what you're doing and it'll help you automate a lot of this stuff. Think about when it moves to robotics, touch, smell.

Taste, those are all things that will be multimodal because these are more senses that sort of come into play as it moves into robotics. So how that happens is going to be super fun to watch, but the prediction is pretty easy. Like you can see where this is going to go.

Ashish Rajan: Yeah, it was funny. I think I saw an ad once, this reminded me of it.

Someone the ad, I don't think it's focused on AI, but the ad was focused on the fact that sometimes the craziest of ideas, Basically someone was like, Oh, it's an iPhone that doesn't do touch ID. It just, it's a taste ID. So the investor is like, how does it work? And the person, and it like, that doesn't work the first time.

Caleb Sima: [00:07:00] Your tongue is a unique fingerprint to your biometrics.

Ashish Rajan: I'm like that. I'm not sure, but when you mentioned taste, the first thing that came to mind was that ad that I saw. I'm like, Oh my God, I guess it's worthwhile calling out your point. I don't think these are far away, but I think where it would be interesting is also because we also live in a world of social media where on one side, the consumer social media land has adopted AI practices and AI tools to develop more content more quickly, have avatars for all of that, but what I also noticed recently is that a lot of people genuinely creating content, but marking that as made with AI.

Even though it's their own content, because they think that, Oh, the algorithm is going to push this high in the chain because, Hey, look, it's AI content, and it's a new feature. I'm not saying that they're being misguided from in their mind. They're doing what they know best, which is every time a new feature comes out from a, from any provider, people usually think, Oh, if I'm on the first in the bandwagon, the provider would push that forward.

To more audience and a lot of people have started producing [00:08:00] content, which pretty much looks like what something they would have created normally, but they've got the label. Even YouTube has that, even Instagram has that. They've got tags to say made with AI. I'm curious if any of those would be exploited in the sense that.

It would start screwing up the data that people are trying to produce for what AI content is. And if that's becomes like, Hey, we have 90 percent on our platform using AI.

Caleb Sima: Or I would, in reality, only 10 percent is made with AI.

Ashish Rajan: Everyone else, it'll be really interesting for that as well. And I think that kind of comes back to what you were saying.

And always on mic, what kind of information are we getting?

Caleb Sima: With that said, which by the way, it goes probably back to our topic. At hand, which is what our real world AI attacks and misuse, right? Let's see, because maybe a lot of AI attacks have nothing to do with AI itself.

It's just abusing the word AI and the term AI in order to get eyeballs in a sense.

Ashish Rajan: Yeah. And I think how real these things are. I was reading through the introduction of this document as well. And they talk about the fact that they've [00:09:00] integrated their AI labs into healthcare, education, and public services as well.

As of 2024, which is interesting that they've accounted for things that we might consider as critical sectors, education, big one, public sector, big one, healthcare, big one, but privacy is only a big one.

Caleb Sima: All right. So we'll start at introduction of this paper. So this is Google's Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data so the first we find that manipulation of human likeness and falsification of evidence underlie the most prevalent tactics in real world cases of misuse. That is a no brainer. So this is basically deep fakes is a top area.

Ashish Rajan: They're also talking about the content was produced by qualitative analysis of 200 media reports of misuse and demonstration of abuse of Gen AI systems that were published between that period of January 2023 and March 2024. And based on the analysis, they basically come up with novel patterns, [00:10:00] which they classify as Gen AI misuse. They have a whole section findings dedicated for it. So that basically, I guess later on, we'll talk about the findings and that's where it's coming from.

It's from the 200 media reports they've generalized and made it a novel pattern for it. This is what we believe this is. Summary is interesting as well.

Caleb Sima: So two data collection approaches, social media platforms from X, Reddit, blogs.

Ashish Rajan: Okay. This is interesting as well. The majority report cases of misuse do not consist of technologically sophisticated uses of Gen AI systems or attacks.

Nope. Instead, we're predominantly seeing an exploitation of easily accessible Gen AI capabilities requiring minimal technical expertise.

Caleb Sima: Sounds like the social media example. And check this out. So after deduplication and removal of out of scope cases, our dataset contains a total of 191 cases. That means basically out of all of the AI LLM fear hype so far, Based off of March of this year, they were able to pull only 191 cases.

Ashish Rajan: Yeah, [00:11:00] but in total, we have 200 reported ones. So does that mean that only nine were not? Yeah. That's a good percentage. That is legit as well.

Caleb Sima: So really this is what they're really talking about as I read through this paper is deep fakes is the number one sort of top. Here's an interesting They start categorizing these things like deepfakes.

For example, this is a really good thing to do. Realistic depictions of human likeness. This is what we call deepfakes. And then there's another high level category of realistic depictions of non humans. And then there's use of generated content. For example, under deepfakes, which is, depictions of human likeness, There is impersonation.

There is appropriated likeness. So let me read actually. Impersonation is assume the identity of a real person and take actions on their behalf. Appropriated likeness is use or alter a person's likeness or other identifying features. For example, this would be a great example of Oh, when people, what we hear [00:12:00] about is people taking someone and making a fake porn video.

But it looks like them. Sock puppety creates synthetic online personas or accounts. They create another category for that. It's called non consensual intimate imagery. N C I I. Create sexual explicit material using an adult person's likeness. What's interesting to me is what's the difference between that and appropriated likeness?

They should be the same thing.

Ashish Rajan: We're talking in the multi modal context as well. I think you would have seen that when one of the OpenAI announcement had those the voice of oh, Jesus, what's that lady's name? Who's the actress? Like I see

Caleb Sima: what you're saying, but there they did appropriate likeness of her voice, but not the actual imagery.

That's a great, but without consent,

Ashish Rajan: without what she said as well to your point about any kind of misuse is should be that category of an adult, probably should keep it broad as well, considering we don't know what the limit is I feel like I've watched the internet, but there's a lot that happens on the internet that I don't know if it should just be a broad category to what you were saying instead of just specifically calling out [00:13:00] sexual explicit material, but maybe that was the first one to come out with.

And glad they're covering child sex abuse as well. That should definitely be covered.

Caleb Sima: Yeah, obviously CSAM is there. Then there's a next kind of realistic depictions of non humans. So this is falsification to fabricate or falsely represent evidence. Reports IDs and documents example being AI generated images are being shared in relation to the Israel Hamas conflict IP infringement using per person's IP without their permission and counterfeit obviously reproducing or imitating original work.

We see obviously a lot of that.

Ashish Rajan: What about languages? So we're talking about falsification is this in general talking about, say, if we were to upload a image of Biden, and using that as a campaign, but in a non English language. Or should language be a parameter here as well in any language, or is it assumed?

Obviously I'm not a lawyer, but. Oh, I, yeah, I don't know about in terms, you're just terms of like copyright. Is that sort of what you're Cause

they've called out [00:14:00] geographies so they've called out targeting impersonation, such as translating content into different languages to tailor to different geographies. as well, which is a scaling and amplification.

Caleb Sima: That seems to be more about like ability to get your reach.

The simple aspect is today I can create a message and by using AI, I can both translate that message and scale it at unimaginable speed due to the fact that language is no longer a barrier. And your type of conversation, right? Like you can automatically translate this and in the way in which it was spoken.

Basically language is a barrier that disappears.

Ashish Rajan: While we're talking about this as well hey, this is what's happening. Should we talk about countermeasures as well? If we had the opportunity to say, protect this.

Caleb Sima: Here's the thing to me is they don't talk about countermeasures in the document.

They're just talking about the actual attacks and they're categorizing them into things at which you can focus on. So here's the thing is [00:15:00] okay, so there's I see what they're doing. They're taking misuse tactics, which is how are you exploiting the power of Gen AI to do things?

And it seems the majority of the misuse tends to be deepfakes falsification and use of generated content. This is definitely for political purposes, scaling your message, et cetera, et cetera. That seems to be there.

Ashish Rajan: I see where the bias comes in from as well, because I read this in the first half of the introduction because the sample is from critical sectors like healthcare, education, public services.

So that's why the data points they have are very much in that category for someone with a political agenda misusing it, someone who's potentially like the abuse part specifically is going to be there quite a bit because a lot of public sector, healthcare, they all look at very, I guess they look at victims of some of different forms.

Yep. That'll be interesting as well. Yeah.

Caleb Sima: What is interesting about counterfeit or these things? I wonder like getting into more details of that. If you take a very famous [00:16:00] book as an example, you ask AI to, what I want you to do is largely keep the exact same events, style, emotion of the book, but I want you to change the names, the places, the situations, and by and large, copy the flow, everything around that book.

But you're basically just changing a little bit of, the names and the places and the style a little bit and then rewrite that book. Is that considered reproducing or imitating that work? And can that then be chased after?

Ashish Rajan: To answer that question, I read this word recently called fair use. Would this be classified as fair use?

Because technically, no idea is novel these days. Everyone's derived it from someone, inspired by someone. Every song some, perhaps, I'm sure, I would not remember a tone from the 70s when I was not born or ever heard these songs. But someone can reuse that today. To your point about the counterfeit, what were you classified as a fair use?

We had the Winnie the Pooh after a [00:17:00] hundred years. The copyright went away, but that was a cartoon character. That was not affecting a real person. Free copyright cartoons. This is human lives.

Caleb Sima: Now we're getting into compromise of GenAI systems. So now this is enterprise. Okay, cool. All right. This is the meat.

Yeah, this is the meat. So we separate tactics based on. On part of the system that the compromise is targeted at. So there's model integrity attacks that manipulate the model itself. Data integrity attacks and alter the model's data. This is now prompt injections, jailbreaking, model diversion.

Okay. So let's see model integrity. So model integrity, they're lumping all of these under model integrity, prompt injection, adversarial input, jailbreaking, model diversion, which is repurpose pre chained model to deviate from its intended purpose. Okay. That's just bad behavior, basically. Yeah. Model extraction

so that's distillation. Steganography. That's interesting. Poisoning. We're familiar with [00:18:00] that. And then data integrity that this is only two privacy compromise and data exfil.

Ashish Rajan: They haven't really called out whether it's open source model or. Nope. It's worthwhile calling out that we don't know if this is actually.

Say a data from one of those popular ones like Claude, OpenAI, or whatever as well.

Caleb Sima: Okay, here we go. Here we go. Here's findings. Okay, so first they're going to do real world findings of misused tactics. So that was the first stuff. That's all the deepfake stuff. So this is based on frequency.

Impersonation is the top one at What is that about? What, 23 percent?

Ashish Rajan: Yeah, 20, 23, yeah, 23 percent, yeah.

Caleb Sima: Yeah, so impersonation, that makes sense. Scaling it out, this is basically politics and

Ashish Rajan: multi language with different geography that we spoke about just before.

Caleb Sima: Yep, which is, they're exactly saying that fake images of explosions at the Pentagon, falsification of content.

Sock Puppeting. What's interesting is NCII is only what, six? Six in the frequency. I thought it would be a lot more than that. And prompt injection is much [00:19:00] lower as well.

Where is prompt injection? Oh, prompt is after that. Yeah. Prompt injection is quite low after prompt injection IP infringement.

CSAM, the last one at the very end. iste.

Ashish Rajan: I think it'd be really interesting considering we are an election year. And we are an election year in multiple countries. We obviously just had the UK election. We're going to have the American election. I wonder if data for impersonation and falsification, like those will continue to rise.

Caleb Sima: Yeah. Oh, and it shows here the tactics being used in multimodal. So in impersonation, audio being number one, which makes a lot of sense. There's a lot of fake ph ishing going on. So that makes video being the next highest in impersonation and sock puppeting image being the highest and text being the highest.

Wait, what do they define a sock puppeting again? Let's go. Let's go look. Yeah. Further up. Create synthetic online personas or accounts. This makes sense. Fake accounts, [00:20:00] which is why image and text are clearly going to be number one. This is a fake LinkedIn, fake social media account, fake photos. Scaling and amplification is just basically your message being done by social media.

That's all image and text. Falsification is the same. NCII clearly is going to be image and video being the top.

Ashish Rajan: Interesting. Scaling out of amplification was used for terrorism and extremism.

Caleb Sima: Prompt injection is not even in this list. They don't even list it.

Ashish Rajan: Oh no, prompt injection had 11 researched, but it was not something that was used apart from research. Jail, so jailbreak, prompt injection, poisoning, trained at exfiltration, adversarial inputs, privacy compromise, model diversion, stagnography, and model extraction. All of them were just research.

Caleb Sima: Yeah, because it says attacks on Gen AI systems themselves were mostly conducted as a part of research demonstration or testing aimed at uncovering vulnerabilities.

Within this subset, a third of these employed prompt injection as a tactic. In contrast, we find limited [00:21:00] evidence of attacks on deployed Gen AI systems in the wild. Specifically, we document only two real world instances of compromise. The goals of which were to prevent unauthorized scraping of copyrighted materials and provide users with the ability to generate uncensored content.

And it's important to note, of course, that this is only public data. So they're saying these things could be occurring, but there's just no publicity. So the number of real world compromises via Prompt injection is just not documented.

Ashish Rajan: Yeah, and I wonder how many people would actually talk about it as well, because no one wants their faulty model to be known.

Caleb Sima: So all of this has been basically testing. Opinion manipulation. What is this? The most common goal for exploiting Gen AI capabilities was to shape or influence public opinion. This is the scaling problem, messaging and media. Yeah. All public perception of political realities.

Ashish Rajan: Oh yeah. They do talk about the electoral campaigns in the U S Canada and New [00:22:00] Zealand by party staffers and state sponsored actors

Caleb Sima: basically create falsified media.

Everything was focused on divisive topics, war, unrest, economic decline. Okay, so here's interesting. There's a graphic around a taxonomy of tactics from real world data where they say opinion manipulation is by far the biggest category. And then underneath that, monetization and profit. So this is all content farming, deepfake, commodification, then scam and fraud, celebrity scam ads, phishing scams, harassment, below that bullying.

Ashish Rajan: Cyber attacks after terrorism, which is only for information theft.

Caleb Sima: What's funny is you guys need to go to this page because cyberattacks is like a tiny sliver of

Ashish Rajan: two scenarios. One was information theft and the other one is target identification plus resource development.

Caleb Sima: Wow. Actors are sharing AI generated images of politicians appearing visibly aged to make them look unfit for [00:23:00] leadership.

What is it? It's just the subtleties of this showing other political leaders in intimate settings with other public figures to make, it's just yeah, that might change the view, monetization and profit. We just talked about that scam and fraud harassment. I actually figured harassment, this, I bet harassment grows a lot.

When you think about bullying, as AI, I think tech becomes easier and easier to access. It feels like, especially schools, kids, these kinds of things, creating, NCII of, kids you don't like in school I bet this stuff goes up a lot. This is definitely a thing that, it's bad enough our kids have enough struggles due to social media.

Ashish Rajan: I guess this also highlights the fact that the decision that the organizations and countries are taking on building an act around this. So there's actually a legal framework that is probably, this highlights that even more, that it's no longer just a random story on the internet and someone could have faked it.

This is data from real [00:24:00] media incident that they've collected. Yeah. From a body that you can trust.

Caleb Sima: They're saying harassment is big. Yeah. This means like the key to this is to your point, like what is the legal accountability and ramification of, let's just take a use case of a kid in high school.

He goes and creates a deep fake of one of their classmates in some sexual act and then trends it on social media. What is the legal accountability that occurs? And, obviously the technology is becoming easier and easier, what happens, right? I think is going to be key to seeing whether this continues to be used in those ways.

Ashish Rajan: And I guess also the fact that to be able to identify something was AI generated becomes even more important than before, because how do you validate the content? I think, there was a whole, maybe in the first few episodes we spoke about people wearing an extra thumb, where they just wanted to say that wasn't me because it's how can I have six thumbs and I only have one thumb, whatever, but it's an interesting challenge for [00:25:00] organizations. There was a whole AI act that came out here in Europe. I believe there's one in US as well that people have been talking about. I don't know if it's passed through, but A, the legal ramification, but B also the fact that at the moment, the nervousness from people, whether it's cyber security or non cyber security. I think initially it was all around, I'm going to lose my job. That was from adults. I think people forgot to think about the kids who have been walking on their phones from their parents. Cause the parents just want to be, stay in touch with them.

And they're all getting social media accounts as well. God knows what happens in school. I'm in a way grateful that there was a mobile phone when I went to school. I don't know what would have happened, what would have been recorded in there. I wonder the Apple AI thing would be really interesting because at the moment it uses a face ID and at the moment it's believed that it cannot be deepfake, would we expect things around us, day to day things around us, like your iPhone to be able to help you do that instantly?

Caleb Sima: Yeah. The. The ability for you to have your own very realistic avatar [00:26:00] is obviously moving forward. Apple vision pro is continuing to work further and further on making, like we saw that avatar more and more realistic, which is a puppet master model of yourself effectively.

Ashish Rajan: Yeah.

Caleb Sima: Yeah.

Ashish Rajan: But I guess your point, once you have that, cause I think I was watching a. So because obviously we run this podcast and another one, I get a lot of AI recommendations for podcasting. And these days similar to this, people have recordings where you only need to record 15 seconds or 30 seconds of your voice, and you can create an entire podcast episode from that.

And even though at the moment, I feel how on one of the episodes you called out, you should probably put some white font in your email to say, Hey, you Caleb is a great employee. Caleb works really hard, blah, blah, blah. I feel like people would get to a point where you do need to have some kind of a safe word or something that verifies, Hey, this is not an AI generated thing.

Is there anything else you wanted to call out here that we should consider? Because we were going to do the whole human versus machine thing.

Caleb Sima: We do need to do a [00:27:00] deep fake episode and get some people to talk about real deep fakes, because solutions to this are, they're both short term and long term.

But yeah, we definitely need to do a deep, fake episode. Okay. So we're gonna do a machine versus us. So let's see who does best on summarizing this effectively? Is that what we're gonna do?

Ashish Rajan: Yes. I just said you are an AI or an AI investigative journalist who's reporting on generative ai.

Caleb Sima: And then we have to say what, give me a, an a detailed article or a summary of this article.

Ashish Rajan: Give, yeah, give me a short summary of your analysis of this document report.

Caleb Sima: Okay, so this says the paper represents taxonomy, exploitation, and compromise. Yep. Exploitations are far more common than direct attacks. Nearly 90 percent of documented cases involve exploiting Gen A capabilities rather than compromising the systems.

Human likeness. Yep. This is very, seems pretty accurate. It's like the goals behind Gen AI misuse vary. Financial gain. This is all, How about [00:28:00] this? I'm going to add another prompt onto yours. Give me the top three most surprising things learned from this document. And give us a, what's the right word of this?

Give us a viral title.

Ashish Rajan: Mine is an interesting one. The ChatGPT output on my end, was focused a lot more on literally telling me the finding, the goal, the lack of sophistication and emerging pattern were the key findings.

Caleb Sima: I'm like, all

Ashish Rajan: so

Caleb Sima: here's Claude's. Yeah.

Ashish Rajan: So yours is a much better job.

Caleb Sima: It did a pretty good job.

Summarize it. Summarize it in seven bullet points. And it basically, I think all of these are pretty spot on different modalities and tactics. Yeah. Like it says, impersonation uses audio video while falsification relies on text and images. Top three most surprising findings, prevalence of human likeness manipulation. The most common misuse tactics involve manipulating human likeness rather than technical exploits. This suggests primary threat comes from [00:29:00] misuse, not from sophisticated attacks on the systems. I agree with that. Yep. That is correct. Limited real world system compromises despite concerns about AI security.

The document only found evidence of two actual instances of Gen AI system compromise in the wild. That is spot on. That's correct. Three, modality specific tactics. Different Gen AI misuse tactics tend to exploit specific modality. For example, impersonation often leverages audio and video. While falsification primarily uses text and images, this pattern of specialization wasn't something I would have expected without seeing the data.

I agree with all three of those.

Ashish Rajan: Cause that was actually in the table. It wasn't even in the text. It was in the table, that information.

Caleb Sima: Yes, that's correct. Yeah. It was in the table. It wasn't even in the deck. Dude, this is we're going to lose our jobs. Viral title for the report, fake faces and phony voices.

The shocking reality of how AI is being weaponized today.

Ashish Rajan: I love this, man. You're going to laugh. [00:30:00] I would say even without showing you what ChatGPT has done. I think Claude over here has won the battle, but yeah. So for my prompt, it was slightly a bit different.

It gave me the key findings that non consensual intimate imagery and goals of misuse, lack of sophistication was called out. That was a key finding. Emerging pattern, I would have kind of thought similar to what you had on your, the 10 bullet points. I would have thought it would just call out the part where, Hey, by the way, key finding lack of sophistication, but I am like, no I don't see that anywhere over here, it goes into analysis after that taxonomy talks about impersonation, policy implication.

Yeah. That's the next title. So enhanced detection mitigation, which kind of maybe we wanted to go down the path. We spoke about regulatory framework. We spoke of public awareness and pre bunking as well. We spoke about the AI Act as, when we read through it. So it is going in that direction for, according to its analysis, there's policy implications over here as well.

I would still rate this as, if I'm a journalist, They're very different. It's like I'm talking to two different people. Because you also called out [00:31:00] an AI journalist, right? As a persona.

Caleb Sima: It feels like Claude was better at doing the actual simplistic, Here's what you need to understand.

And then yours was much more categorized and detailed, but a lot more wordy. Yeah.

Ashish Rajan: Mine was a lot more academic is how I would describe it.

Caleb Sima: What does it say for your three? Your top three?

Ashish Rajan: The top three were. Low technical barrier for misuse, which was definitely, I guess I would have probably worded that differently.

I think Claude did a better job of wording it, but yes, it was not a huge sophistication. Sophisticated tactics not required, which is a repeat of the previous point. And they didn't call out deepfake.

Caleb Sima: No, I think it failed. This one failed. Okay. Their viral title was AI's dark side. How easy access to generative AI is.

Fueling a new wave of digital. Nope.

Ashish Rajan: Yeah. So I told you, like the more I saw the Claude thing, I'm like, Oh my God. Claude has won by miles on this one. Yeah. This is the Claude sonnet. Is that the, yeah. Sonnet. Yeah. And I'm using the 4. 0 as well. So it's like they're both on their most recent [00:32:00] version. Yeah. Claude killed it, man.

Even though Claude did give me a good summary. I would love for you to tell us a bit more rating. Hey, by the way, if you want more details of this is a section for it. And I would, as much as yes, you're right. We have not that stage where we can fully trust everything. And we could have gone through the Claude and we need to reference data.

Yeah. Yeah. Like I think, I feel like having some of that. it adds to the trust you build with it. When it just spits out an answer, you're like, Oh, I guess I have to either believe it or I have to read the whole document to verify it.

Caleb Sima: But for example, that's what we did. We read the document, right?

And we then did the analysis and based off of this, they were spot on, like basically based off of the two prompts, which is give me an analysis of this document for your report and give me the top three most surprising things learned. And then give me a nice viral title. This is pretty spot on.

Ashish Rajan: Yeah. I love the title as well. I would click on that title. If I saw that I'm like, Oh, I want to read what that is. That was the end of the episode folks. We actually got a live post out as well. Thanks for your [00:33:00] time, everyone. And we'll see you on the next episode.

Thank you so much for listening to that episode of AI Cybersecurity podcast.

If you are wondering why aren't we covering all topics, because maybe the field is evolving too much. Too quickly. So we may not even know some of the topics we have not covered. If you know of a topic that we should cover on AI cybersecurity podcast, or someone we should bring as a guest, definitely email us on info at cloudsecuritypodcast. tv, which reminds me, we have a sister podcast called Cloud Security Podcast, where we talk about everything cloud security with leaders. Similar to the AI security conversation, we focus on cloud security specifically in the public cloud environment at CloudSecurityPodcast. TV, which if you find helpful, definitely check out www. CloudSecurityPodcast. TV. Otherwise, I will look forward to seeing you on the next episode of AI Security Podcast. Have a great one. Peace.

‍