Ecosystem Studies of the Best Proxies Services (Series 6)

[weitang]

Overview of residential proxies services

By penetrating Proxy Residential services, the author was able to effectively collect information about the pool of residential proxies. Specifically, during the penetration process, the author launches multiple Residential Proxies to capture data from different locations and accounts each day and runs them at different times of the day to reveal as much as possible about the state of the dynamic Residential Proxies pool. Ultimately, the author captured 6 million residential proxies by sending 62 million requests. Due to the volatility of the network environment (especially in mobile networks), this number should be considered as an upper limit for the number of residential proxies hosts. The table shows the distribution of each residential proxy service provider across different network blocks and ASes, with ProxyLite having the largest pool of rotating residential proxies, followed by Proxies Online and ProxyRack.

When analyzing the countries, ASNs, and ISPs with the largest number of residential proxies, the author observes a clear long-tail distribution phenomenon. A handful of countries, ASNs and ISPs contribute the majority of residential proxies. For example, although ProxyLite is headquartered in Hong Kong, China, the majority of its residential proxies come from Europe, which may be related to the web censorship in that country. It is worth noting that ProxyLite claims to have 72M+ proxies, a huge amount.

Residential Proxies Infrastructure and Services

While researching the infrastructure of residential proxies services, the author found a series of hidden back-end gateways between front-end gateways and residential proxies. These back-end gateways were found in the connections between the proxies' gateways and the residential proxies. I will explain what follows: a series of hidden back-end servers exist between the front-end gateway and the residential proxies. From the perspective of the residential proxies, these servers can be seen as gateways, hence the author refers to them as back-end (hidden) gateways. The author's traffic logs, PUP traffic, and passive DNS datasets record these connections. The author observes that prior to traffic relaying, PUP-hosted residential proxies typically communicate with hidden back-end gateways rather than interacting directly with front-end gateways. The author further analyzed the PassiveDNS data and found that these hidden back-end gateways share a set of IPs with the front-end gateway, strongly suggesting that they belong to the same network architecture. Specifically, using ProxyLite as an example, the author observed that the PUP hosting residential proxies always communicate with the back-end servers rather than the front-end gateways before relaying the infiltration traffic. And some of its subdomains act as back-end gateways that communicate with the residential proxies, while all front-end gateways are clean. This shows that by separating the different components in this way, it makes the whole ecosystem more robust.

The author also investigates the sharing relationship between different residential proxies service providers. By calculating the crossover rate and setting strict criteria, the author found multiple residential proxies addresses shared across service providers that may have a partnership or some of these brands actually belong to the same company.

In a further analysis of the residential proxies infrastructure, the author discovered through port scanning that these front-end and back-end gateways tend to have a large number of contiguous ports open. The presence of these ports not only allowed the author to identify the infrastructure of residential proxies services, but may also provide clues to potential detection and defense measures.

Conclusion.

The complexity of the residential proxies ecosystem is manifested on multiple levels, including cooperation between service providers, distribution of infrastructure, and involvement of IoT devices. By delving into the infrastructure and operations of residential proxies services, the author has revealed the true state of their network. the transparency and trustworthiness of ProxyLite's services can be examined as a major player in the field. In future research, it will be important to explore how to effectively detect these networks of proxies.