Chicago’s WindyGrid Puts Open Data to Work – InApps Technology 2022

With its WindyGrid project, the city of Chicago is using open data from multiple sources, both internal and external, to offer an unprecedented view of everything happening within the city, as Tom Schenk, Chicago’s chief data officer demonstrated at the recent MongoDB World 2017.

WindyGrid combines data collected in more than three dozen of systems within city departments — such as 911 calls, non-emergency 311 calls, building permits, health inspections — and combines it with data from other sources, such as weather data and tweets, to produce a comprehensive view of the city. It can show where police, fire and ambulance vehicles are in real time. It can plot reported potholes and the status of each complaint.

The city’s philosophy is that technology should be driven by people who use it and benefit from it. It’s not just about making it faster, he said, but how to do something interesting with it.

“We didn’t just want to create a fancy map, but use it to drive operations,” Schenk said.

For instance, the city has only 36 food inspectors to monitor 16,000 food establishments covering the city’s 235 square miles. WindyGrid’s visualization tool plots food inspections on a map. Users can click on each dot to read results of each inspection. But they also can filter by failed inspections, which he demonstrated as a heat map.

By combining data such as whether an establishment has an alcohol license, past failures, and reported sanitation issues in the area, the public health department can focus resources on businesses more likely to have issues.

The health department also traps mosquitoes for DNA testing in 189 locations around the city as it tracks West Nile virus. WindyGrid can display as a bubble map locations where the most mosquitoes test positive for the virus, which the city can use to try to get ahead of the problem.

Flexible Schema

Originally released in 2011, WindyGrid is used in city departments such as police, fire, public health and streets and sanitation.

Originally built with a Flash front end and MongoDB on the back end, a 2.0 update in 2015 switched the front end to HTML/JavaScript, and it was open sourced as OpenGrid. The city releases a new version of WindyGrid every two to three months, and consistently pushes out new features, such as the bubble maps, which were released two weeks ago. Those features are then added to OpenGrid.

The data in WindyGrid is based on what the city departments say they need. The Mongo flexible schema makes it easy to add new datasets as managers say they need them, Schenk said. Hyper-local weather information from Dark Sky is pulled in every 20 minutes from 90 locations around the city.

Inspections are an area of particular focus, Schenk said in a separate interview, because of the huge caseload its inspectors face.

“We think we can systematically address inspections and predict where to best use resources. Those type of systematic research questions we can build into WindyGrid, and as people get used to it, serve up the analytics — which isn’t a new thing. Tableau will do forecasting for you, create a time-series… but we understand the business-case problems, where the pinches are in the city, and we understand analytics, and we can better map those two things together,” he said in a separate interview.

Adding Features

The city is working with a number of partners, including the University of Chicago, Argonne National Laboratory, and the University of Illinois-Chicago, on adding features.

An important one Schenk said he’d like to add is the ability to press “play”:

“Rather than a static map, what if you could see a two-week window? That’s very important when it comes to flooding,” he said.

You might see that all the calls started coming in at 5 o’clock when people calling in with reports of flooding.

“I don’t just want to see what happened that day, I want to see how the day developed, or the past week,” he said.

The city’s Office of Emergency Management and Communications uses this data to coordinate response to flooding. It can be different from a neighborhood where there are known issues, while there might be something happening in a neighborhood without those issues that are causing an uptick in calls, he explained.

He’d like to use more natural language processing in search of the data, and the city is working on improving predictive analytics in a substantial way behind the scenes.

“We call it putting a brain behind WindyGrid,” he said.

“Rather than real-time data and historical data — it’s just storing data in a database and reaching it. We are interested in knowing what data is going to be in the future. We’re working to make this as a service,” he said.

“Those type of systematic research questions we can build into WindyGrid, and as people get used to it, serve up the analytics — which isn’t a new thing. Tableau will do forecasting for you, create a time-series… but we understand the business-case problems, where the pinches are in the city, and we understand analytics, and we can better map those two things together.”

WindyGrid, for instance, can predict where there’s going to be an uptick in rodent complaints — and areas where there likely are problems, but for whatever reason, people aren’t reporting it.

It has an active project on predicting the homes where children are most likely to be affected by lead paint.

City workers use WindyGrid with their manager’s approval.

OpenGrid, meanwhile, works from publicly available data published from the city’s open data portal, where for the past five years, the city has been publishing data such as crimes, business licenses and building permits. The two datasets largely overlap.

“We take that app code and layer it on top of the open data portal so you can see data in that same non-siloed way,” he said. The architecture was designed so you can swap out the back end.

OpenGrid can be configured to run off of a variety of sources, including Plenar.io, which can ingest data from Socrata and Ckan open data portals.

Developers can match the OpenGrid interface to work with different database engines or APIs. OpenGrid’s Leaflet-based map makes it easy to integrate with tile maps to add additional map data, such as OpenWeather layers.