Internet "censorship" is often associated with repressive governments
filtering the traffic of their citizens, but it goes well beyond that.
Internet service providers sometimes filter—or alter—the
traffic that they carry, companies restrict employees based on keywords and
URLs, courts naïvely order certain URLs to be blocked, and so on. But it
is difficult for any particular internet user to know just what it is they
can't get at. That problem is what the Tor Open Observatory of Network Interference (OONI)
project is hoping to help solve.
The overall goal for the OONI project is "to collect data which shows
an accurate representation of network interference on the Filternet we call
the internet", according to the web site. One obvious, though time
consuming, way to do that is to gather information from multiple different
"locations" on the internet, and that is what OONI has set out to do. Of
course, the OONI project itself can only reach out so far, so the intent is
to enlist other participants—essentially "crowdsourcing" the data
There are other internet censorship tracking projects—Google's Transparency Report
and Herdict for example—but the OONI
README notes that other efforts either use a closed methodology or closed
software. As befits a Tor
project, though, OONI is fully open source. No top-level LICENSE file for
the moment, but one would guess it will be similar to Tor's permissive license.
The core piece (ooni-probe) is written as a framework in Python,
with an eye toward
contributions of additional tests (called "plugoos") and reports. "Tests"
are meant to detect
censorship events by comparing the results obtained locally with some kind
of experimental control. That control could be obtained via the Tor
network, for example, or via some other means. The tests can use various
kinds of "assets", which might include lists of URLs, IP addresses and
ports, or keywords, as their input. Current tests
include checking that Tor bridges are functioning, determining whether HTTP
filtering is occurring, checking for DNS tampering, doing address and port
scans, detecting Squid proxies, and so on.
While there are plenty of tests that could be added, seemingly the area
needing the most attention
right now is the "reports". Currently, test failures are
essentially just written to an unstructured text log file, which can
be stored locally or uploaded to a server. Tools to interpret the data and
to provide higher-level visualizations of the types and locations of
internet censorship are planned.
While the OONI code is under heavy development, the project can
already claim some successes. ooni-probe was used to detect eight
blocked web sites for internet users in Bethlehem, West Bank. The
probe scanned more than one million sites and found that users are blocked
from eight news sites "whose reporting is critical of
[Palestinian Authority] President Mahmoud Abbas".
ooni-probe found that T-Mobile USA's Web Guard "feature" blocks
access to much more than the advertised categories. In particular,
sites for Tor, the Internet Archive WaybackMachine, Chinese sports news,
French economics and financial news, a Japanese URL shortener, and many
others, were blocked though they didn't fall into any of the listed categories: "Alcohol,
Mature Content, Violence, Drugs, Pornography, Weapons, Gambling, Suicide,
Guns, Hate, Tobacco, Ammunition".
OONI is just getting started, but it is clearly a welcome addition to the
internet landscape. In order for John Gilmore's famous quote ("The
Net interprets censorship as damage and routes around
it"—which seems to be an informal slogan for OONI) to be
true, the internet, or really its users and operators, must be aware of
where that censorship is occurring and how it is being applied. With tools
like OONI (and the others, though it's unclear why they aren't more
transparent), routing around that censorship will be easier. The free flow
of information on the internet depends on being able to do so.
to post comments)