Big Data and the Internet of Things

How Open Data is Transforming Cities

Font Size:
Default font size
Larger font size

Steve Hansen is a councilmember in Sacramento, Calif.

Stephen Hardy is CEO of mySidewalk, a technology startup using spatial data to improve community decision-making in 180 U.S. cities.

Tom Shenk, Jr., is chief data officer for the city of Chicago.

Posted: Wednesday, March 15, 2017 6:16 pm

According to technology lore, the “Internet of Things” first came into being in the early 1980s when someone in the computer science department at Carnegie Mellon University connected a soda machine to the Internet so they could tell without leaving their desk whether their favorite brand of soda was still in stock and had reached the desired temperature.

Since these auspicious beginnings, humankind has been adding almost every conceivable device to the global network.

From refrigerators and HVAC systems to pacemakers and space probes, there seems to be no limit to the way in which inanimate objects can be configured to monitor, control and report on just about anything, from just about anywhere.

Environmental monitoring can provide amazing benefits: sensors under bridges can provide real-time data on river levels to trigger early warnings of flash floods; devices attached to buoys in the ocean can sense a tsunami and send an alert; air quality sensors can reduce asthma admissions at local hospitals; in-line water quality monitors can automatically divert the flow when contaminates are detected; wired bridges can report slippery surfaces in time to summon the salt trucks; cisterns with embedded chips can drain themselves when rain is in the forecast; and bridge crossing gates can close when flooding occurs upstream.

Sensors of all kinds can be employed to report almost anything: pot holes, leaking water mains, heavy traffic, flooded roads, gun shots fired, highway crashes, trains on the wrong track, power outages, spikes in electricity consumption, low product inventories, wet basements, burned out streetlights, noise pollution, malfunctioning stop lights, strong winds and oppressive heat. We could go on.

Some might say, “The Internet of Things has given local governments and institutions wonderful new tools for delivering services, mitigating disasters, conserving resources, reducing accidents and preventing crime.”

While others might say, “Big brother, terrorists and nefarious hackers now have frightening new tools for spying on citizens, picking our pockets, manipulating our thoughts and launching attacks.”

Is a “smart” parking meter a great way to maximize enforcement efficiency, calm traffic congestion, reduce parking fines and provide convenience to the motoring public? Or, could it be used to track the location and movements of private citizens? Could hackers steal account information; could the system fail in bad weather; is it fair to those who don’t have smart phones and credit cards?

Perhaps the devil is in the detail.

The trick, experts say, is to start with good intentions, engage stakeholders, communicate the benefits, take security and privacy concerns seriously, and be as transparent as possible with the collected data.

When done right, “all these devices that are sensing where we are, what we’re doing and how we move, will help us build better cities,” said Steve Hansen, a Sacramento, Calif., councilmember. “In Sacramento we’ve started with pedestrian counters in downtown parks, so we can actually see the volume of traffic when it comes to justifying investments in those parks…. And these pedestrian counters have also helped us lease office space in our Capital Mall area, because we know now what the foot traffic looks like.”

Hansen, who spoke at the recent New Partners for Smart Growth Conference in St. Louis, said the combination of smart parking meters and traffic control devices have allowed Sacramento to better handle all the additional traffic when a new arena was built downtown. Visitors can reserve a parking space in advance, via cell-phone app, and traffic lights can be adjusted remotely as needed to prevent bottlenecks before and after big events.

Computer chips placed in garbage bins allow Sacramento sanitation crews to plan their routes and go back for any missed containers.

“If you’re a city official you know that the biggest complaints you get are about garbage and parking,” Hansen said. “If we’ve suddenly taken away the top two categories of public complaints, can you imagine how the level of satisfaction in government has gone up? …That’s like curing cancer in the world of civics,” he joked.

Stephen Hardy, CEO of mySidewalk, a technology startup using spatial data to improve community decision-making in 180 U.S. cities, said in his plenary address at the conference, “It’s not really the machines that are interesting; it’s the ‘data exhaust’ they kick off, and what we can do with that.”

Hardy said the winners in the new economy will be those who control and take the most advantage of data. This, he said, applies to communities as well as businesses. “In the information age, most cities will protect their data, many will open their data and a few will actually harness that data,” Hardy said.

Hardy introduced a simplified way that data from three different sources can be overlaid to make decisions. Where the three data streams intersect, the answer can be found. Here are some small-scale examples:

And here are some larger-scale examples:

As can be seen by several of these examples, data gathered by one government agency or city department can sometimes be used by another to solve a completely different problem. And, if that data is opened up in a secure way to the entire community, it can be used to solve problems that officials did not even know about or imagine.

Tom Shenk, Jr., is chief data officer for the city of Chicago and he oversees the city’s open data portal. He said making the city’s data available to the public is a good first step toward providing transparency and giving citizens the tools to better understand problems and come up with their own solutions.

Data shared by Chicago includes potholes reported through the city’s 311 system; red light camera violations, speed camera violations, problem landlords, public chauffeurs, bike-sharing trips, restaurant health inspections, crime data, and taxi trips taken, among others. Some of these datasets include millions of rows of data, Shenk said.

The most popular data in the portal is information about the salaries of city employees and a record of every purchase the city makes, Shenk said. That’s good for transparency, but other data is valuable to researches, college students, economic developers, and businesses of all sorts.

“Open data provides a means to create an ecosystem around data which includes multiple stakeholders and initiatives that extend beyond transparency,” he said. “The data portal becomes a town square…. It allows people to come together and meet other people with shared interests… converse and then go their own way.”

He said a local group of programming hobbyists began meeting a few years ago to discuss the data and how it could be used. That group, Chi Hack Night, has since grown to include “thousands of designers, academic researchers, data journalists, activists, policy wonks, web developers and curious citizens,” according to the club’s web site. The group has created several helpful apps using the city’s data and has become a large, vibrant and productive civic community, Shenk said.

The group developed SweepAround.Us, for example. Using open data, this service alerts individuals to street sweeping activity by providing email, text or calendar alerts. was developed to easily find flu-shot locations across the city. The code was created by a volunteer and is open source, so it’s been adopted by Boston, Philadelphia and San Francisco, Shenk said.

Chicago has partnered with numerous universities and organizations on developing various hardware appliances that have been implemented to sense and report data to the portal. The University of Chicago and partners have built the “Array of Things,” which includes about 30 sensors hung on utility poles around the city. It reports sound and vibration information, climate and environmental data and its low-resolution infrared cameras measure sidewalk temperature to determine if sidewalks need snow removed or salt applied. The code is open-source and the data is available to everyone in the city, Shenk said.

“This is not just for scientists, it’s not just for the city,” he said. “Imagine you’re an 8th grade teacher and now you can use real-life examples of humidity from a sensor down the street to understand humidity in your area from around the park or around the lake.”

The city of Chicago has created the website to allow city departments and the public to create maps using various data sets. The city uses the maps to develop pothole repair plans, predictive rodent extermination plans, and food-borne illness prediction models, among other things.

In one pilot study, by crunching the numbers on food-borne illnesses, the city was able to shave an entire week off the average time it took to shut down restaurants with critical violations, when compared to the status quo approach, Shenk said. Some of the data points used to predict violations included:

1) Restaurants with previous violations;

2) Three-day average high temperature;

3) CDPH risk level;

4) Location of restaurant;

5) Nearby garbage and sanitation complaints;

6) Type of facility;

7) Nearby burglaries;

8) Whether the establishment has a tobacco or liquor license;

9) Length of time since last inspection;

10) Length of time the restaurant has been inspected.

By finding critical violations faster, the city has reduced the number of patrons who become ill, which in turn reduces medical expenses, lost time at work, and even some fatalities, Shenk said.

More about

More about

Online Poll