Skip to content

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Reduction of Active Domain Namespace on the Internet

DOI

Background and Hypothesis

The Internet and Web 2.0 used to function as a distributed store of knowledge, incentivized by advertising and facilitated by search engines. Large Language Models (LLMs) are actively disrupting this model by centralizing information storage and access. Consequently, free collaboration and knowledge dissemination (e.g., Stack Overflow) are drying up.

Furthermore, training dataset curators are hiring high-caliber individuals to directly source their expertise and transfer it into LLM weights. We hypothesize, therefore, that online activity is focusing on a few LLM-driven platforms at the expense of wider exploration across diverse Internet domains.

Methodology

To assess this hypothesis, we analyzed the monthly rankings of the top 1 million domain names from February to November, comparing the results between 2024 and 2025. The ranking data is sourced from the Tranco List.

In these rankings, leading domains by DNS activity typically exhibit relatively low churn. For instance, a domain in the 20th position one month might drop to the 30th in the next, but it is highly unlikely to plummet to the 96,000th position for example. Conversely, churn in the long-tail section of the ranking is expected to be substantial. Because the differences in activity levels decrease further down the list, it is quite plausible for a domain at position 312,000 one month to transition to 516,000 or so in the following.

We leverage this gradient in rank volatility as follows. We calculate how many leading positions ($k$) we need to compare – using the same number $k$ across two adjacent months – to reach a certain symmetric difference between the two sets (i.e., the number of domain names that appear in only one of the two sets). This symmetric difference is a cut-off on the volatility gradient. It cuts a kernel – the number of the most actively accessed domains $k$ from long-tail noise. We arbitrarily set this cut-off to 10,000 (1% of the dataset). As long as this number is kept constant, month-on-month kernels can be reliably compared.

Implementation

Because the Tranco ranks are conveniently structured with one domain name per line, it is trivial to conduct this analysis using Bash. The repository contains two scripts:

  • pull: Downloads the monthly ranking datasets.
  • kern: Calculates the kernel between adjacent months with a precision of 1,000.

Results

Our analysis shows that the active namespace began to shrink from the summer of 2025 ($p$ = 0.015). Comparing the second halves of 2024 and 2025, there is an indicative 8% decrease in the number of domain names with comparable activity.

Shrinking Active Namespace

About

Supplemental material

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages