Skip to content

Nearly Half of All Internet Traffic Comes From Bots in 2023

Bots are automated programs designed to perform various online tasks. In 2023, bots account for a staggering 47.4% of total internet traffic. That means almost half of all activity on the internet is not even human. So how much of this bot traffic is helpful, and how much is harmful? As an experienced cybersecurity professional, let me break down the fascinating world of bots on today‘s internet.

Good Bots vs Bad Bots: A Deeper Look

The internet bot ecosystem can be divided into two main categories – good bots and bad bots.

Good bots follow the rules and perform beneficial jobs like indexing web pages, monitoring websites, gathering data, and automating clerical work. According to Imperva, these helpful bots account for 17.3% of internet traffic in 2023.

Bad bots, on the other hand, break the rules and engage in malicious activities like content scraping, credential stuffing, fake account creation, spreading malware, DDoS attacks and more. Imperva estimates these shady bots make up 30.2% of online traffic – nearly double the percentage of good bots.

Let‘s explore some specific examples of good and bad bot types in more detail:

Good Bots

Search Engine Bots

  • Search bots like Googlebot crawl the web and index web pages to enhance search engine results. Googlebot alone generates over 50% of all good bot traffic.
  • Other common search bots include Bingbot, Yahoo Slurp, and YandexBot.

Monitoring Bots

  • Performance monitoring bots like those from Pingdom, Datadog and Nagios regularly check websites to ensure they are up and accessible.
  • These bots alert web administrators about any downtime issues so they can be quickly fixed.

Social Media Bots

  • Social bots automate activities like posting updates, analyzing user data and providing basic customer service on platforms like Twitter and Facebook.
  • They may also identify policy violations and moderate harmful content.

Chatbots

  • Chatbots simulate human conversations and interactions on messaging platforms and websites. They provide instant 24/7 responses to user questions.
  • Common examples include customer service chatbots and personal assistant bots like Siri.

Web Scraping Bots

  • Some web scraping bots follow site policies and scrape data for legitimate purposes like research, price monitoring and market analysis.

Bad Bots

Fake Account Bots

  • These bots automatically create fake accounts on sites and services like social media platforms and online games.
  • They allow their operators to inflate followers/friends counts, spread propaganda, and more.

Web Scraping Bots

  • Unlike good web scrapers, these bots violate site terms by aggressively scraping content, pricing data, and other assets without permission.

Credential Stuffing Bots

  • Credential stuffing bots take lists of stolen usernames/passwords and input them on other sites to compromise accounts through brute force.

Ad Fraud Bots

  • Ad fraud bots generate fake ad clicks and impressions to siphon money from online advertising networks.
  • They cost the industry over $100 billion per year according to some estimates.

Scalper Bots

  • Also called sneaker bots, these snatch up limited concert tickets and hot products like gaming consoles to resell at inflated prices.

Spam Bots

  • Spam bots distribute unsolicited bulk emails, ads, and messages on platforms like social media and messaging apps.

DDoS Bots

  • DDoS bots overwhelm websites and online services by flooding them with traffic to take them offline.

SQL Injection Bots

  • These bots probe websites for vulnerabilities in order to inject malicious SQL code and steal or destroy data.

This overview demonstrates the wide range of both positive and negative applications for internet bots in 2023. But how much does each category actually contribute to overall traffic?

Bot Traffic Percentages and Statistics

Bots have become an increasingly large portion of internet activity over the past decade. Let‘s examine some key statistics on bot traffic percentages:

Bot Traffic Percentage of Total Internet Traffic

Year% From Bots
201545.6%
201638.7%
201742.2%
201837.9%
201937.2%
202040.8%
202142.3%
202247.4%

Good Bot Traffic Percentage

Year% From Good Bots
201817.5%
201913.1%
202015.2%
202114.6%
202217.3%

Bad Bot Traffic Percentage

Year% From Bad Bots
201820.4%
201924.1%
202025.6%
202127.7%
202230.2%

Analyzing this data reveals a few interesting trends:

  • The total percentage of internet traffic from bots has increased over the past 8 years, from around 38% to 47% as of 2022.
  • Good bot percentages declined from 2018 to 2020, but are once again on the rise at 17.3% in 2022.
  • Bad bots have grown significantly from 20.4% of traffic in 2018 to 30.2% in 2022.
  • The gap between good vs bad bot percentages has widened, with nearly twice as much traffic coming from bad bots compared to good bots.

These statistics illustrate the increasing scale and sophistication of automated bot programs, especially those used for malicious purposes.

Identifying Bot Traffic and Behavior

To protect websites and applications from malicious bots, it‘s crucial to identify bot traffic and behavior patterns accurately.

Integrated analytics platforms like Google Analytics and Heap provide breakdowns of bot vs human traffic. Unusual spikes or drops in traffic, bounce rates, conversions and other metrics may also signify bots.

Specifically, bad bots tend to display these anomalous activity patterns:

  • Sudden traffic spikes from unexpected regions
  • Massive jumps in pageviews
  • Abnormal changes in bounce rates
  • Unusual variations in session duration
  • Suspicious transaction patterns

Examining the user agent string can help reveal a bot‘s purpose and origins. For example, the user agent for Googlebot is:

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Tracking the frequency and geographic distribution of requests can also detect bots violating normal human patterns.

However, advanced persistent bots (APBs) mimic human behavior patterns more realistically using machine learning and other evasion tactics. More intelligent bot management solutions are required to detect these stealthy threats.

Bot Mitigation Strategies and Best Practices

With bad bots accounting for nearly one third of all internet traffic in 2023, implementing effective mitigation strategies is crucial for businesses. Here are some recommended tactics and best practices:

Bot Detection Systems

  • Use dedicated bot detection systems like PerimeterX which analyze behavior patterns in real-time to identify and block malicious bots.
  • Combine machine learning, IP reputation data, JavaScript challenge screens and other signals for accurate detection.

CAPTCHAs

  • CAPTCHA challenges help distinguish humans from bots by testing abilities like audio recognition and image labeling that are difficult for bots.
  • However, some sophisticated bots can now solve CAPTCHAs with over 70% accuracy. Using CAPTCHA alternatives is recommended.

Blacklisting Known Bad Bots

  • Blacklisting IP addresses and user agents associated with malicious bots blocks them at the perimeter before reaching websites.
  • Maintain an up-to-date blacklist, as bot operators frequently change these.

Rate Limiting and Throttling

  • Rate limiting and throttling restricts the number of requests bots can make to deter denial of service attacks and aggressive scraping.

Web Application Firewalls

  • WAFs offer advanced bot protections including rate limiting, CAPTCHA integration, whitelisting/blacklisting and machine learning driven threat detection.

User Behavior Analysis

  • Analyze page scrolling, mouse movements and other user actions on your site. Bots exhibit more suspicious behavior than real humans.

Multifactor Authentication

  • MFA stops many automated bot login attempts, as they cannot complete additional steps like SMS code verification.

API Protection

  • Secure APIs against bot attacks using API gateways to authenticate access, limit request rates, and block suspicious API calls.

User Education

  • Educate customers not to click on sketchy links which spreads malware bots. Teach employees best practices to avoid bot infections.

A layered defense combining multiple technical and policy controls is most effective against the growing bot threat landscape. Ongoing monitoring and adaptation is also essential as bots become more advanced.

The Future of Bots Online

Bots are firmly entrenched into today‘s digital world, supporting both beneficial and harmful applications. Based on current trends, bots could account for over 50% of internet traffic within the next couple years.

As a cybersecurity professional, I expect bots leveraging artificial intelligence and machine learning to become commonplace. AI-powered bots are already being used for video game testing, data mining, speech recognition and other tasks.

However, these same technologies will enable more sophisticated bad bots capable of incredibly realistic human impersonation. Expect a rise in personalized, targeted social engineering attacks. Bots will also become adept at circumventing many current anti-bot solutions.

To stay ahead of these threats, the cybersecurity community must invest in next-generation protection powered by AI and deep learning itself. The tech industry should also establish security and ethical frameworks around bot development and usage.

While bots present increasing risks, their upside cannot be overlooked. Used properly, bots automate mundane work and enhance productivity. The key is cultivating more good bots, while keeping the bad ones at bay. If we build a robust, resilient and multi-layered defense, the internet can continue thriving with bots and humans working side by side.

nv-author-image

Michael

Michael Reddy is a tech enthusiast, entertainment buff, and avid traveler who loves exploring Linux and sharing unique insights with readers.