OONI-Research Reports
Introduction
The Open Culture Foundation (OCF) began reorganizing the OONI project in October 2023 and opened the ooni-research project on the foundation's Github page. The project's planning details, research directions, data extraction programs, etc., are presented in an open-source and open spirit manner!
OONI Observation Data and Internet Autonomous System (ASN) Coverage." OONI primarily collects detection data of local networks through volunteers and participants to observe whether there are instances of significant interference with the internet. OONI is a project under the Tor Project, and from its inception in 2012 to 2023, it has observed data from nearly 2 billion sets, covering 241 countries, with observations across 30,000 networks.
And we are also curious about the current status and patterns of observation data in Taiwan (TW). How much observation data comes from known Autonomous System Numbers (ASNs)? Are there still Internet regions that we are unaware of and remain unobserved? Ultimately, we want to examine the data to question whether our internet is truly free.
With these questions in mind, we begin to plan and analyze OONI observation data. After the OONI detection application is executed, the observation data is uploaded to AWS S3 storage (Registry of Open Data on AWS). Therefore, it is possible to download the raw observation data through existing package tools for organization and analysis. For detailed file download procedures, please refer to the 'ooni.py' in the project folder.
Observation Data
Data fetched from September 1, 2023, to December 3, 2023, spanning a total of 94 days, with approximately over two million observation records. The minimum time interval for data is hourly. Currently, there is a data contribution for every hour, achieving a 100% coverage rate for hourly observation data.
| Month | Days | Hours |
|---|---|---|
| 2023/09 | 30 | 720 |
| 2023/10 | 31 | 744 |
| 2023/11 | 30 | 720 |
| 2023/12/01 ~ 2023/12/03 | 3 | 120 |
| Total | 94 | 2,304 |
The total observation hours are counted on the basis of ASN (Autonomous System Number), with unique ASN in each hourly record. The data count represents the number of observations checking websites. The average observation count is calculated as the data count divided by the total observation hours. (For data organization, refer to the provided reference.)
| Month | Total Observation Hours | Data Count | Average Observation Count |
|---|---|---|---|
| 2023/09 | 2,342 | 712,724 | 304.32 |
| 2023/10 | 2,512 | 1,029,845 | 409.97 |
| 2023/11 | 2,243 | 945,011 | 421.32 |
| 2023/12/01 ~ 2023/12/03 | 240 | 90,509 | 377.12 |
| Total | 7337 | 2,778,089 | 378.64 |
ASN Data Patterns
From the observation data collected from September 2023 to December 3, 2023, there are a total of 32 unique ASNs. Among them, AS3462 and AS18041 contribute approximately 78.94% of the observation data. (For data organization, refer to the provided reference.)
| ASN | 2023/09 | 2023/10 | 2023/11 | 2023/12/01 ~2023/12/03 |
Total |
|---|---|---|---|---|---|
| AS3462 | 328,656 | 470,142 | 468,767 | 42,663 | 1,310,228 47.16% |
| AS18041 | 41,567 | 415,974 | 386,789 | 38,415 | 882,745 31.78% |
| AS18419 | 72,405 | 74,721 | 62,593 | 7,195 | 216,914 7.81% |
| AS38136 | 2,900 | 3,100 | 3,000 | 300 | 9,300 0.33% |
| AS1659 | 3,100 | 1,001 | 4,053 | 300 | 8,454 0.30% |
| AS945 | 400 | 3,100 | 3,000 | 300 | 6,800 0.24% |
| AS24158 | 1,511 | 1,520 | 627 | 598 | 4,256 0.15% |
|
Others (Excluding other ASNs without observation data each month.) |
|||||
| 2023/09 | 2023/10 | 2023/11 | 2023/12/01 ~2023/12/03 |
SUM | |
|---|---|---|---|---|---|
| Total (with records each month) | 450,539 | 969,558 | 928,829 | 89,771 | 2,438,697 87.78% |
| Total (Others) | 262,185 | 60,287 | 16,172 | 738 | 339,392 12.22% |
| Total | 712,724 | 1,029,845 | 945,011 | 90,509 | 2,778,089 100% |
Complete List of ASNs with Observation Data
| Month | Unique ASN |
|---|---|
| 2023/09 | 20 |
| 2023/10 | 22 |
| 2023/11 | 19 |
| 2023/12/01 ~ 2023/12/03 | 10 |
| 2023/09 ~ 2023/12/03 | 32 |
In the current registration of ASNs in Taiwan, there are a total of 437, and the observation data (TW) with unique ASNs for only 7.32%, which is a relatively low proportion.
| AS945** | AS9678 | AS18182 | AS38136** | AS131668 |
| AS1659** | AS9916 | AS18412 | AS38841 | AS134823 |
| AS3462** | AS9919 | AS18419** | AS38844 | AS147049 |
| AS4780 | AS9924 | AS19527 | AS41378 | AS396982 |
| AS7539 | AS17421 | AS24157 | AS55720 | |
| AS9416 | AS17716 | AS24158** | AS131596 | |
| AS9674 | AS18041** | AS24164 | AS131627 | |
|
||||
|---|---|---|---|---|
Actions
In the OONI Explorer database, Taiwan's observation data ranks among the top 10; however, an analysis of the data content reveals that approximately 80% of the current observation data is concentrated in the contributions from AS3462 and AS18041. During the analysis period, ASNs contributing data each month represent only 7.32% of the known ASNs registered in Taiwan.
At this moment, the observation data is not comprehensive and diverse enough to fully reflect the current network patterns in Taiwan, including those of the three major telecom operators, broadband services from cable TV, fixed network services, and second-class telecom services (virtual mobile network services), etc.
As one of the OONI community partners, the Open Culture Foundation is responsible for local promotion and observation. In the upcoming promotion strategy, efforts will be adjusted to assist contributors in understanding ASNs and discovering observations from unknown ASNs. Continuous provision of straightforward installation guides will also be maintained.
Reference
- Observation Data and Data Compilation Document(Google Sheets)