The challenges of supporting geolocation in WordPress

As much as we get addicted to mobile phones and online services, nobody (outside of cyberpunk fiction) actually lives online. That's why maps, geolocation services, and geographic information systems (GISes) have come to play a bigger role online. They reflect they way we live, work, travel, socialize, and (in the case of natural or human-made disasters, which come more and more frequently) suffer. Thus there is value in integrating geolocation into existing web sites, but systems like WordPress do not make supporting that easy. The software development firm LuminFire has contributed to the spread of geolocation services by creating a library for WordPress that helps web sites insert geolocation information into web pages. This article describes how LuminFire surmounted the challenges posed by WordPress and shows a few uses for the library.

LuminFire developer Michael Moore presented the library, called WP-GeoMeta-Lib, at a talk (the slides are available in Moore's blog posting) on August 16 at FOSS4G, the major open-source geolocation conference. FOSS4G's success itself demonstrates the growing importance of geolocation, as well as the thriving free-software communities that create solutions for it through group projects such as the Open Source Geospatial Foundation (OSGeo). FOSS4G held its first conference in 2007 in Thailand. Its global wanderings, which would require sophisticated geolocation tools to track, brought it this year to Boston, where it topped 1,100 registered attendees—its biggest turnout yet.

With so many GIS projects aimed at the web, such as the popular Leaflet JavaScript library, why try to do geolocation through WordPress? LuminFire developed its library to satisfy requests from its clients, but Moore threw us some statistics to show how important the project would be to the larger public. One estimate claims that 28% of the world's web sites use WordPress. Thus, a good GIS service for WordPress can vastly increase the availability of geographic information.

So what are the problems? The data you store for web pages—including geospatial data—is called metadata by WordPress. It stores this data as plain text. The existing GIS plugins have to manipulate geospatial information as text, or convert it back and forth between text and a native format, which is all cumbersome and slow. Furthermore, WordPress uses MySQL for its storage. I believe this bolsters the popularity of WordPress, because MySQL is easy to use and adequate for typical web needs. But it has a limited geospatial model. Although both MySQL and its fork, MariaDB, have made strides adding spatial support, its spatial analysis capabilities are paltry compared to PostgreSQL. The PostgreSQL geospatial extension, PostGIS, is an anchor of the free-software GIS movement, and could be found all over the FOSS4G conference.

Discussions at WordPress about supporting other databases serve mostly to show how daunting a port to PostgreSQL would be. Although MySQL and PostgreSQL both adhere to some standards, they differ in significant ways (for instance, in how they implement the essential feature of automatically incrementing columns). A PostgreSQL plugin for WordPress was created, but it never worked well and is now outdated to the point of being unusable. So WordPress administrators don't really have the option of using PostgreSQL or PostGIS.

Moore mentioned one example of the problems with MySQL: up through version 5.5, it determined whether geometric objects overlapped by checking each one's bounding box (the smallest rectangle that can contain the object). For instance, two circular objects that are close together may be considered overlapping even if just their bounding boxes overlap. This is inadequate for geolocation, which requires more precise processing. MySQL improved the code for version 5.6, but the majority of sites still run version 5.5 or earlier.

Moore also lamented the lack of Python support in WordPress, because Python offers so many powerful geolocation tools. WordPress is based on PHP, which I believe is another reason for its popularity. PHP doesn't offer geolocation support, but it doesn't seem to get in the way of what LuminFire wants to do.

A final requirement for LuminFire's development effort concerned its users: few of them are GIS experts who can understand the software popular in the GIS community. LuminFire wanted a tool that ordinary WordPress administrators could use. Given these limitations, LuminFire chose to develop a library for WordPress instead of a plugin. This WP-GeoMeta-Lib library, distributed under GPLv2, uses the WordPress API.

Working with a system that doesn't understand your data

In an email exchange after the conference, Moore gave me an in-depth explanation of the techniques used by WP-GeoMeta-Lib to turn the limited support by WordPress and MySQL into a platform for efficient and accurate location data. Here, we provide some background for people who want to understand WordPress's system of hooks and storage; it may also be of general interest because it illuminates how developers can deal with the tradeoffs posed by the platforms they work with.

At the center of WP-GeoMeta-Lib is a set of tables in the MySQL database that parallel the four WordPress tables holding metadata. WP-GeoMeta-Lib creates the parallel tables to store geospatial data. Instead of the MySQL data type used by WordPress for its metadata (LONGTEXT), WordPress uses the GEOMETRYCOLLECTION data type, the most appropriate type in MySQL to store geospatial information. To carry out basic geographical inquiries, such as whether a location is within a larger region or what the distance is between two locations, you need subclasses of MySQL's GEOMETRY type. Using LONGTEXT for that purpose would be like to trying to perform calculus using Roman numerals. GEOMETRYCOLLECTION is a good general-purpose type that recognizes points, lines, boundaries, and other concepts that are basic to geography. The generality of GEOMETRYCOLLECTION makes it useful for storing the arbitrary mix of geometric elements that different applications require.

WP-GeoMeta-Lib uses the WordPress hook system to intercept calls to the WordPress API and substitute a geospatial database for the WordPress metadata tables. Let's say a developer retrieves data from the MySQL database using the WP_Query object. (In theory, the developer could bypass WordPress and run raw SQL, but WordPress discourages this.) The function that handles WP_Query calls an internal WordPress function named get_meta_sql() on every query, so WP-GeoMeta-Lib registers its hook for that function. Every time get_meta_sql() runs on that WordPress site, it calls all the functions that the site developer passes to the hook. When the developer's functions finish, the function behind WP_Query picks up and continues as if nothing had intervened.

WP-GeoMeta-Lib uses hooks into the four types of metadata—posts, users, terms, and comments—that are part of the filters in the Plugin API. Plugins can add metadata to WordPress, delete metadata, update it, and retrieve it. By hooking into these low-level functions, WP-GeoMeta-Lib is guaranteed to have a whack at anything that affects WordPress metadata. And this metadata is where it's most convenient for the user to store structured data, such as geolocation data, in WordPress, so WP-GeoMeta-Lib works in a way familiar to WordPress developers.

The WP-GeoMeta-Lib hooks check the WordPress call for references to the more than a hundred spatial functions supported by MySQL. If it finds a spatial function, WP-GeoMeta-Lib changes the SQL to find the data of interest in its own geospatial metadata instead of the general-purpose WordPress metadata. The library also alters that fragment of SQL to run the requested spatial function. After updating the SQL query, the library returns from the hook so WordPress can do its usual stuff with all the elements of the API call.

Thus, WordPress doesn't have to know what WP-GeoMeta-Lib is doing. WordPress goes ahead and stores its own version of the geospatial element in the LONGTEXT fields. It would be hard to tell WordPress to alter its usual behavior, so WP-GeoMeta-Lib does not try to suppress the LONGTEXT elements. It just ensures that its own metadata is used for geospatial elements—the LONGTEXT metadata is basically inert.

Part of Moore's design for WP-GeoMeta-Lib is based on a humble assumption that something might go wrong during its run, and he wanted to minimize the chance that it would interfere with the function that calls it, or other functions that website developers might pass to hooks. Therefore, he chose to hook in his functions at the latest possible point.

Moore used WP-GeoMeta-Lib to develop plugins that are even easier to use than API calls for certain tasks. Two of these, Brilliant Geocoder for Gravity Forms and GeoMeta for ACF, allow WordPress sites to use popular form plugins to give end-users tools for viewing and manipulating geospatial data. They also help administrators put WP-GeoMeta-Lib to direct use, as we'll see.

Example use: searching by location

A LuminFire client in the medical services industry uses WP-GeoMeta-Lib to enable customers to search for doctors by location. Information about these doctors is stored in a directory exposed through WordPress's custom post type, which allows the developer to create new fields that WordPress hasn't provided. This particular client made the doctor's name the title of each custom post, and created a collection of custom fields to store information that patients might use to search for doctors, such as their specialties and qualifications.

Because the client used ACF Pro to store each field, the developers used the GeoMeta for ACF plugin to create a field for the doctor's location. When the medical site's staff edit listings of doctors and add or update an address, the GeoMeta for ACF plugin geocodes the address to get the coordinates of the office location. The coordinates are returned in GeoJSON format. When all the doctor information is saved by the generic ACF plugin, the GeoMeta for ACF plugin invokes WP-GeoMeta-Lib to store the coordinates in the WP-GeoMeta-lib metadata tables.

Customers can then log into the site and view the doctor directory, which allows searches using various criteria (called a faceted search by ontologists), including the customer's address and a maximum distance. A search using these facets triggers a function that geocodes the address and runs a spatial query against the WP-GeoMeta-lib metadata. The results are passed back to WordPress and the directory is redisplayed with only the doctors located within the maximum distance.

The resulting code is both robust and extendable. The medical company plans to add more complex searches, such as finding doctors who restrict their practices to certain regions or counties. Although the most basic distance-based search could have been carried out using custom queries and trigonometry, the more irregular searches will need the full, sophisticated spatial query capabilities.

Example use: connecting users by geographic location

A non-profit uses WP-GeoMeta-Lib as part of coordinating waterway stewardship among cities, watershed management organizations (WMOs), Watershed Districts (WDs), and volunteers. Pollution in one area of a watershed affects all downstream areas, so the WMOs and WDs are responsible for regulating and managing pollution across multiple city and county borders. This client runs several types of spatial queries, and stores location information in several places.

For instance, pollution mitigation features (such as a rain garden) are usually group projects. The volunteer who launches the initiative wants to search for other volunteers who live close by. And after the project is complete, both the city and the relevant WMO want to know about it.

When volunteers sign up, they enter their information through a form powered by the Gravity Forms plugin. Hence Moore's Brilliant Geocoder for Gravity Forms plugin, which the non-profit use to convert the volunteer's address into a location. As with the previous example, data about the projects is stored on a custom post type, and the function that stores the data uses GeoMeta for ACF to let volunteers and site editors enter project locations.

Finally, the site created a custom category collection for projects so that a project can be tagged with a WMO or WD. Each of the tags for a WMO or WD includes its boundaries as geospatial information. This could be used in the future for applications such as finding a WD that would benefit from a project, finding all volunteers living within the boundaries of an WMO's region, or checking which regions benefit from funding. In short, this data will enable spatial-based reporting and mapping, giving the non-profit the opportunity to add powerful geospatial enhancements to their site over time.

Although mapping is a big part of GIS, Moore pointed out that neither of the cases mentioned in this article use mapping currently. A lot of searching and reporting is also location-based and is difficult or impossible without spatial data.

Conclusion

WP-GeoMeta-Lib uses a lot of workarounds. It is clever in its use of parallel metadata, but it requires developers to build new geolocation tools on top of MySQL, somewhat reinventing the wheel when other environments use PostGIS or proprietary tools. The example uses highlighted in this article show the potential for GIS tools in a web environment; hopefully this potential will draw more developers to create them.

The basic assumptions behind WordPress would probably make it hard for WordPress to upgrade to more native support for GIS. For instance, it would have to allow arbitrary objects as metadata instead of plain text. Moore's goal of bringing geographical information to a wide audience of administrators and users is a commendable one, and WP-GeoMeta-Lib seems to have accomplished a lot toward that goal.