Dive into the world of cybersecurity and automated traffic with key insights from Imperva's 11th edition Bad Bot Report.
The world of cybersecurity never ceases to amaze with its intricacies and challenges. One of the ongoing battles that organizations face is the constant threat posed by bad bots infiltrating the digital landscape. In a recent interview with Sean Martin and Erez Hasson from Imperva, key insights from the 11th edition of the Bad Bot Report were unveiled, shedding light on the evolving nature of automated traffic and the impact it has on various industries.
Unraveling the Bad Bot Landscape
The conversation kicks off with Sean Martin introducing the topic of bad bots and the significance of Imperva's Bad Bot Report in providing insights into the world of automated traffic. Erez Hasson, a senior product marketing manager at Imperva, dives into the details of the 11th edition report, which is based on a staggering 6 trillion blocked bad bot requests processed by the Imperva network over the past year.
Delving into Key Statistics
Erez Hasson elaborates on the critical statistics highlighted in the report, such as the percentage breakdown of automated traffic into bad bots and good bots. The report categorizes bad bots based on their sophistication levels, ranging from simple to advanced (evasive), emphasizing the need for robust bot management strategies to combat sophisticated attacks.
Industry Insights and Use Cases
The conversation shifts towards exploring the impact of bad bots across different industries, with a focus on sectors such as Law, Government, Travel, Airlines, Retail, and Financial Services. Erez emphasizes the need for organizations to understand the sophistication level of bot attacks targeting their industry to effectively mitigate risks and safeguard their digital assets.
Transforming Data into Action
Sean Martin underscores the importance of translating the insights from the Bad Bot Report into actionable strategies for organizations. By leveraging the educational content provided in the report, companies can enhance their understanding of bot-related challenges and tailor their security programs to address potential threats effectively.
AI's Role in Bot Evolution
The discussion moves into the intersection of artificial intelligence (AI) and bot activity, highlighting the increased use of AI-driven attacks, including credential stuffing attacks orchestrated through AI algorithms. The evolving landscape of automated traffic poses challenges for organizations, necessitating a proactive approach to mitigate risks associated with bot-driven activities.
Safeguarding Against Bot Abuse
The conversation touches upon the misuse of bots targeting AI interfaces, leading to increased operational costs for organizations. Additionally, the resurgence of debates around the legality of web scraping underscores the complex nature of combating bot-related activities and protecting proprietary content from illicit scraping practices.
Conclusion
As the conversation draws to a close, a call to action is extended to readers to delve into the insights provided by Imperva's Bad Bot Report and equip themselves with the knowledge needed to combat bot threats effectively. The collaboration between security teams, leadership, and practitioners is essential in implementing robust bot management strategies to safeguard against evolving cyber threats.
By understanding bad bots and automated traffic, organizations can bolster their cybersecurity defenses and stay ahead of malicious actors looking to exploit digital vulnerabilities. The insights shared in Imperva's 11th edition report serve as the base of awareness, guiding organizations towards a more secure digital future.
Learn more about Imperva: https://itspm.ag/imperva277117988
Note: This story contains promotional content. Learn more.
Guest: Erez Hasson, Product Marketing Manager at Imperva [@Imperva]
On LinkedIn | https://www.linkedin.com/in/erezh/
Resources
Learn more and catch more stories from Imperva: https://www.itspmagazine.com/directory/imperva
View all of our RSA Conference Coverage: https://www.itspmagazine.com/rsa-conference-usa-2024-rsac-san-francisco-usa-cybersecurity-event-infosec-conference-coverage
Are you interested in telling your story?
https://www.itspmagazine.com/telling-your-story
Unveiling the World of Bad Bots: Insights from Imperva's 11th Edition Report | A Brand Story Conversation From RSA Conference 2024 | An Imperva Story with Erez Hasson | On Location Coverage with Sean Martin and Marco Ciappelli
Please note that this transcript was created using AI technology and may contain inaccuracies or deviations from the original audio file. The transcript is provided for informational purposes only and should not be relied upon as a substitute for the original recording, as errors may exist. At this time, we provide it “as it is,” and we hope it can be helpful for our audience.
_________________________________________
[00:00:00] Sean Martin: Hello everybody, this is Sean Martin. I'm coming to you live, recorded live anyway, on location from RSA conference. And, uh, as you know, if you listen to my show, Redefining Cybersecurity, I get to talk to cool people about cool stuff. And this is one of those topics that I love talking about. I forget how many we've done of these now, where we get to look at the bad bot reports from Imperva.
Uh, no question. The industry looks to this report to understand what's going on. Uh, with respect to bad bots and, and uh, automated traffic on the internet and, and Pro does an amazing job with us and I'm thrilled to have Arez on to, uh, talk about the 11th edition. Let's style this up to 11, Arez.
[00:00:41] Erez Hasson: Yes, please.
[00:00:41] Sean Martin: How do we do that?
[00:00:42] Erez Hasson: Yes. Yes. So first of all, we're gonna have some fun. Yeah, . So thank you for having me. As you said, uh, 11th edition of the Bad Bot Report. I'm super excited to be talking about it. I'm a big nerd for the topic of bots and bot management. Um, I think it's a very unique and a privilege for me to be able to work on this report that we've done for the 11th year now.
Really goes to show how many years we've been covering this topic of automated traffic. Um, and this year's report actually is based off almost 6 trillion blocked bad bot requests that went through the Imperva network throughout the past year. So it's a lot of data to, to dive into. Um, and yeah, we can dive into some key stats, uh, if you want.
[00:01:26] Sean Martin: Yeah, absolutely. Let's give folks a bit of, well, a quick word about you and your role on this, because I think that that's an important piece.
[00:01:33] Erez Hasson: Of course. So first of all, Erez Hassan, I'm a senior product marketing manager at Imperva, a Thales company. I am working on our bot management product and our client side protection product.
And this has been the fourth year I've been working on this report. And just on a personal note, it's exciting for me every year to, because again, that data is working together with the threat research team. And every year I'm excited to see what is it going to look like? Because obviously as a person who, studies this throughout the year and sets a bunch of Google alerts on bot, bot management, bot attacks, and gets a bunch of articles every day.
I come into this with a set of predictions, expectations maybe, but eventually the data is the data. So it nice to see whether or not it clicks with what I've predicted or doesn't. So, yeah.
[00:02:21] Sean Martin: Yeah. So there's a traffic, there's a threats in the traffic, there's the impact, the customers, I'm sure we're going to talk about all that stuff.
Um, let's start with. Maybe an overview of the last 11 years, anything stand out to you in terms of where things started 11 years ago, um, where they sit now, is there much more? I presume there's a lot more bot based traffic on the web.
[00:02:46] Erez Hasson: Bots have come a long way. I mean, scraping has existed for as long as search engines were essentially available.
Uh, but the more nefarious, more malicious use cases, more, I would say, some walking on the grey line use cases that are not specifically illegal, but also not very nice to legitimate users of the internet, have been around for a couple of years. Some interesting trends that happened throughout the past ten years was, uh, That at certain years, uh, bad bot traffic was around 20 to 25 percent.
There was a certain year where, uh, I believe it was 2014, that, uh, scraping by search engines have been very aggressive. And then you've seen A large amount of traffic actually came from good bots. But what we're seeing in recent years, I think, is kind of settling down of the trends because there was a certain year where still a lot of new internet users have now been able to actually use the internet.
So mobile devices have become much more common across the world. Internet connection has become much more common across the world. So there were a couple of years where you saw the human traffic just above 60%, Fairly high as compared to what we see it today, which is, it's, it's almost a 50 50 split today.
So it's 49. 6 is automation or the 50. 4 is human. But there were a couple of years when new internet users were kind of shifting that towards the human.
[00:04:09] Sean Martin: So it's 50 percent me, the human, 50 percent my co founder Marco, the machine. I think that's, we'll go with that now. But I, I want to talk about, um, why that 50 percent and people might wonder, I think, For folks in security who look at applications, they might say, I know why.
But, I have a feeling it's the applications and machine to machine, app to app API. Stuff is a big part of that.
[00:04:32] Erez Hasson: No, absolutely. I mean, the proliferation of APIs has changed the game in the recent, uh, I would say two to three years, we're seeing much more of that automated traffic targeting, uh, APIs. So to give some of the high level stats from the report, which I think matters, obviously the big ones that we just kind of cover that people always ask about it's the, the, what percentage of traffic was automated.
So we said it's 49. 6 and you can take that and break it down into 32%, which was actually bad automation or bad bots. And then around 18 percent were good bots. So, you know, the scrapers that Google uses just to index web pages or even price comparison websites, as long as it's, uh, okay with retailers to, uh, Scrape prices and provide users with the best prices.
So that's part of the good automation. And then the bad automation is scraping that's, uh, not, uh, allowed. And then some other examples are account takeover, which is obviously more malicious in nature, trying to take over user accounts using bots in different techniques. Uh, that's the high level stuff.
And then when you're talking about bad bots, we also classify those by the level of sophistication. Essentially, that sophistication means how well, or not at all, they try to evade being detected on the internet. So, what techniques they employ in order to evade being detected, so for that we have three different categories, which is simple, moderate, and advanced.
The simple ones, obviously, most likely are scripts that are not even trying to hide their identity or evade being detected. Moderate ones, you add to that some more techniques like headless browser, puppeteer, stuff like that, essentially browser automation. And then the more advanced ones usually have a set of techniques that includes browser automation, um, rotating proxies like residential proxies, mimicking mouse movements, uh, defeating captures and, and stuff like that.
So 60 percent of those 32 bad bots, uh, 32 percent are actually either moderate or advanced, which we like to call evasive because they're the ones actually, I would say self aware and trying to evade being detected.
[00:06:35] Sean Martin: It's fascinating, this space. And I want to, um I want to talk a bit about maybe some scenarios.
I don't know if these have changed over time. I can, I can recall, I don't know, a good, a good 10, 11 years. Uh, the, the airline industry being one, one competitor targeting another to buy seats and fill, basically sell off the plane without actually paying for any seats and made them go out of business if I'm not mistaken.
So that's like from 10 years ago or so. Where have things gone recently? Because I just had a chat with, with Nani Singh.
[00:07:09] Erez Hasson: Yeah.
[00:07:09] Sean Martin: And we were, she was painting a scenario around a restaurant. Where a restaurant has an application that, that's an API driven menu, that then's an API driven ordering system. Mm hmm.
How, is that a scenario that could be,
[00:07:22] Erez Hasson: I have a simple equation that I like to say is that essentially high demand with low availability equals bought interest. So whatever has a high demand and low availability.
[00:07:36] Sean Martin: Love hearing.
[00:07:37] Erez Hasson: It's been consistent. So whether it be the restaurants they were just brought up, whether it be concert tickets or sport sport events, tickets or sneakers, you know, they're always drop these limited edition sneakers that there's like, like a first come first serve.
But what is a first come if the bot's first, you know, . So that's, that's a, that's an issue. Um. Around 2020, we saw PlayStation 5s also because there was a chip shortage and it was very difficult to get your hands on a PlayStation 5. So bots were running that market for a good year. You couldn't get one without at least trying to go through like third party marketplaces and get those for almost sometimes even double the price.
So yeah, anything that has a high demand and the travel industry in particular is always an interesting one. Because there are a bunch of use cases that are very relevant to that industry. Um, as you mentioned, competitors looking at your inventory, your seat inventory, you're trying to beat you at the market, so they're scraping your prices.
Um, there are also specific metrics that the industry cares about, like a look to book ratio. It's essentially, The ratio of people who actually booked the flight from just looking at it. So having bots just scrape it for availability messes up that ratio for those teams that care about that. The marketing team.
[00:08:51] Sean Martin: That's an algorithm that changes how well their programs are doing or yeah, it changes the price.
[00:08:56] Erez Hasson: And of course you have the loyalty program. So account takeovers for trying to grab miles off accounts and, and things like that, and you have seed spinning is another unique one to the industry, which talks, uh, is essentially about.
Sort of like inventory hoarding, so grabbing seats using bots, just holding them so legitimate users can't get those seats. And then you're denying legitimate users, but you're also hurting the bottom line of that airline because no one's actually booking that flight, ending up with empty flights.
[00:09:25] Sean Martin: So you mentioned earlier that some of this may not be illegal, and I can probably paint a couple scenarios in my mind where you want this to be available.
I think airlines want their inventory and their pricing available for aggregators and brokers to have access to it, right?
[00:09:44] Erez Hasson: Right.
[00:09:45] Sean Martin: But then the bots have access to it too, and so you want it to be available, but only to some people, legitimate people. It's not illegal for some of the stuff. Unethical, maybe close to illegal.
So how does that that weirdness of, we need it that way, we don't like it that way, but it's And we can't really do anything legally to protect ourselves. So how does that happen?
[00:10:06] Erez Hasson: It's a really good question and it's one of the reasons why I like being in the field in bot management specifically because it's not a clear cut security use case.
So I mean, there's a lot of different teams in the organization that are involved or that it's impacting, as I just said, like marketing teams, they care about their conversion rates, their ratio, look to book. So there's a bunch of teams involved, um, in it and, um, This just makes it much more challenging when you're facing a threat that's, as you said, it's not illegal.
I'll just give you a small anecdote. I've been working on a deck and I've been looking at some online websites that sell bots. It's not the dark web. It's just there. You can search for it and you will find it. And they'll strictly say, oh, we can defeat bot management solutions. We'll give you rotating residential proxies.
Specifically, actually ones that talk about, like, buying certain sneakers. Or ones dedicated to booking restaurant appointments. They're all there on the internet and it's completely It's, I don't know if it's illegal, but obviously it's not illegal. So it's, it's right there. And then what that means is that organizations have to fend for themselves.
[00:11:16] Sean Martin: Yeah. Yeah. It's unreal. It's unreal. And I wanted to, um, I don't know if you have any other stats over the 11 year. I mean, it's impressive to have 11, 11 years of this data, any other industries that stand out or regions or any use cases that, that, uh, come to mind?
[00:11:34] Erez Hasson: Sure, I can say one thing and I'll talk into the industry specifically is that there's one thing I also like to say working on this report for a couple of years now is that it's not only the level of automation that you see at a certain industry.
There might be an industry like the gaming, I believe, had 57 percent of traffic, uh, come from bad bots. You should also be looking, there's a second chart in the report that just as I said, 60 percent of bad bots were either moderate or advanced. So there's a chart that talks specifically about the sophistication level for each industry.
And I personally believe that this is a more telling sign of your bot problem as an industry or as a business than just the level of, uh, automation that's on your website. Because the more advanced that traffic is, that means the more persistent they are and they have a much better reason to target you and they're not going to go away if you do not have a solid bot management strategy.
So some of those industries that had the high, uh, ratio of sophisticated bots Uh, Law and Government, to no one's surprise, obviously. The Travel and Airlines, also very high. Um, Retail, Financial Services. So that is, uh, by the industries, you can figure out that there is a solid incentive for attackers to target these specific industries.
[00:12:45] Sean Martin: Yeah. The intent and the, uh, the capabilities make it, make most people an easy target. I think, uh, I don't know that we're still an issue of, or having an issue of we're not a target. Most companies saying that, I think most. Realize they probably are, but this data proves.
[00:13:02] Erez Hasson: Exactly. Yeah, it's a common, we don't have a bot problem.
It's a common thing. Yeah.
[00:13:08] Sean Martin: So let me, let's, let's go here. I want to, a lot of research here and maybe, maybe you want to tie in some of your research team, uh, capabilities as well. And, cause what I'm, what I'm picturing is an amazing set of data from an incredible group of people. That's interesting. But also valuable to an organization to say, I operate in this space, I have these use cases, I have these types of technologies in place, and I see this kind of traffic, therefore I probably have an issue with X, Y, and Z related to this report.
So I can envision an organization, security team, leadership team, practitioners, and even some of the business leaders saying, here, let's look at this report and understand what's going on. How we map to this, and therefore we can change our security programs and our, perhaps our, our security processes as well to mitigate some of these challenges, mitigate some of these risks.
So, how does that play in from a customer perspective as you're working with them to translate the report, basically as a short answer question, translate the report into action for the company?
[00:14:21] Erez Hasson: Yeah, um, first the report, as you said, it offers the industry data, but it doesn't stop there. I think one of the things that's important to me with this report is that it becomes an educational piece for these specific personas.
I think everyone that's handling or having the problem with bots in one way or another is that the report covers everything from defining bots, to what is a good versus a bad bot, how even a good bot could be a problem. For example, if you're a marketing persona, a company, even good bots skewing up your analytics data can affect your decision making and you might be spending money where you shouldn't be spending money because you thought that's valuable traffic, but it was just automated traffic regardless of its nature.
So there's the impact of that. And then other than the industries and talking about how industry, uh, each industry suffers from it. There's a bunch of charts that I always like to include, uh, at the appendix, which talk about the different use cases, the telling signs of those use cases, the impact, and what industries it usually affects, because some bot use cases, they cross the industry.
So if you have a login endpoint, it doesn't matter what industry you are, you're going to be a target of an account takeover attack. And if you have some financial incentives behind user accounts, that probability is just higher. Um, so that's, there's that. And then the chart that talks specifically about industries and the use cases.
So there's a lot of things, uh, that can be, uh, done to educate or, uh, just explain to customers, like, what are the different problems that they might be facing.
[00:15:50] Sean Martin: Yeah, awareness and understanding visibility and then the ability to take action. Um, Top of mind, not just at the conference, I think in general, is a two letter acronym that most people can't get away from talking about in any conversation.
So I'm sorry I'm going to take us there. I want to look at this from two different perspectives. One is the creation of bots and activity driven by artificial intelligence. Maybe some of your thoughts on that, things you're seeing. The data and the research, perhaps even from customer conversations.
[00:16:28] Erez Hasson: Yeah, sure.
So as I said, every year when I start working on this report, I'm, I'm excited cause I have my set of expectations and predictions and I've been looking into that throughout the past year and I had some predictions in mind and it's, I don't know if it's nice. I mean, for me, it's nice to see that they kind of came, came, came true, uh, for, for the market.
Maybe not, but, uh, what, what I thought would happen and we're seeing that we're going to see an increase in automated traffic for two reasons. The first you mentioned the use of AI for non technical users or less technical users to ask for it to help you write a script and, and, and similar things. And we're actually seeing attacks being orchestrated and launched through AI.
And the interesting thing about it is that it's also, it's kind of reflected in the data in the report as well. Because what I said is that you, yes, you're going to see more bots, but I think it's going to skew metrics into more simple bots. Because essentially what, at least the capabilities as they stand right now, It's, it's, it'll help you maybe create a simple script for a bot or things like, it's, it's, you're not going to create like an all in one bot.
It's, an all in one bot is certainly, it's a bot that's able to, uh, go through an entire process of, let's say, buying a sneaker. So it would put the right item in the cart, uh, go to the checkout, complete that checkout, even defeat a capture for you. So these are very advanced bad bots, and that's not the capabilities that there are there right now.
So we're actually seeing. More simple bots in our data than we did before. So it's a 6 percent increase from last year. And, and that is associated with that. And the threat research team right now is working on research around attacks that were being launched, um, using AI. And one of those particularly as are actually credential stuffing attacks, so it can't take over.
Yeah, so that's, there's definitely that aspect. There's an impact still.
[00:18:13] Sean Martin: Even simple doesn't mean non impact.
[00:18:15] Erez Hasson: Exactly. And, and as I said, it's just the beginning. We, we don't really know. We can speculate. Uh, we had interesting speculations of even launching a bot attack. It's, again, it's just speculating, but launching a bot attack and then launching that 1, 000 times on different websites.
And then analyzing that data and figuring out, you know, AI helping you figure out what blocked you. And then you come the other day and try and synchronize that, whatever stopped you. So that's just like an assumption, but yeah.
[00:18:42] Sean Martin: So serious stuff like ATO and even just still bad denial of service, if they're taking traffic away from legitimate stuff.
So the other thing I wanted to do as we wrap here is, um, the use of bots to target AI. So organizations presenting their interfaces for customer support, um, Marco and I were looking at this for some of our own use and thinking, well, if non humans come and interact with our AI, CHATBOT, it's not cheap to have an NLM service at your disposal.
People using it, it can run the cost up. So that's an example where I think bots can misuse and abuse an NLM interface and cost company money, if not more. Create issues for the legitimate users again. So Ana, any thoughts or any signs of that happening?
[00:19:38] Erez Hasson: That's actually a very interesting use case that I can definitely see a case for.
Cause it reminds me of what we're seeing of bots targeting APIs and raising API costs for organizations. Cause now they have to pay for each API call that was bot driven, actually. So they're losing money for that. So I can definitely see a case for bots targeting AI that's embedded in websites. Um, And that definitely requires protecting those, those paths.
So there's definitely a case for that. And then one other thought I had about AI, that's the increase in automated traffic is also because of the large language models using scraping in order to train these models. So that's another thing that we're seeing. It's also resurfacing the debate around the legality of web scraping, because some websites do not want their proprietary content scraped, for example.
So there's a lot of debate around that going on.
[00:20:32] Sean Martin: Triple whammy.
[00:20:33] Erez Hasson: Exactly.
[00:20:35] Sean Martin: Oh boy, it's uh, it's an incredible space and I'm sure you have a lot of fun and I'm grateful that you do what you do. Because there are numbers, thousands, hundreds of thousands of companies that rely on that work to uh, to protect themselves from uh, from the bad stuff that's uh, hitting the network traffic.
So Rez, I want to uh, want to thank you for giving us an update on the 11th year. Congratulations on on that nice, uh, Nice 11th run and, uh, cheers to the Imperva team and the research team there at Imperva that, uh, pulled all this together and helped customers, uh,
[00:21:10] Erez Hasson: Thank you very much and thank you for having me.
[00:21:11] Sean Martin: Thank you. And thanks everybody for listening and, uh, I encourage everybody to grab a copy of the Bad Bot Report, the 11th edition from Imperva. Have a read, understand, learn, and apply, uh, apply some protections because, as we noted, your industry is certainly seeing something here if you don't recognize it yet.
So hopefully this will give you some insight in what to look for and, uh, be sure to connect with the Impera team. Thanks everybody.