Census Reporter

  • Archive
  • RSS
  • Ask me anything

More ways to use Census boundaries on your slippy map

A few weeks ago we released a simple feature to add Census boundaries to your “slippy maps.” Today we’re happy to share some more tricks of this nature. The earlier feature only gave you all shapes of a given summary level. Now you can be more selective—introducing CensusReporter.GeoIDLayer As before, this code depends upon LeafletJS.. In fact, we’ve renamed the javascript library to make that clear. The old one is still there, but if by chance you’re already using it, you should update the javascript src url. To use this, first load LeafletJS and then our library on your page:

<script src="http://censusreporter.org/static/js/cr-leaflet.js"></script>
Then, assuming your map is called map, this is all you need to do to add the boundary of Illinois to it:
new CensusReporter.GeoIDLayer('04000US17').addTo(map);
Under the hood, the GeoIDLayer uses our /geo/show JSON api, so you can pass any arguments that understands. Specifically, you can pass in a comma-separated string of geoids, and you can also use our “containment” syntax to get all geographies of a certain kind within a given parent. (See below for an example) GeoIDLayer extends L.GeoJSON, so you can pass an options object as the second argument to control style and interaction. By default, clicking on layers added using this layer will open the appropriate Census Reporter “profile page” for the geography. You can disable this by setting the autoclick option to false. Here’s an example which shows how to add all states in the Midwest division to the map, with a popup which shows their names:

var layer = new CensusReporter.GeoIDLayer('040|02000US2',{
    autoclick: false,
    onEachFeature: function(data,layer) {
        layer.bindPopup(data.properties.name);
    }
}).addTo(map);
Once you’ve created the layer, you can add and remove more geoids easily:

layer.addGeoID('04000US06'); // add California
var illinois = layer.removeGeoID('04000US17'); // remove Illinois
Because a L.GeoJSON object is really a layer group, calling removeGeoID can return the shape you’re removing as a Leaflet layer. You could add it back later with:

layer.addLayer(illinois); // restore Illinois
This isn’t something we actually use directly in CensusReporter, so we welcome your feedback about whether it’s useful, or what could make it better. Even more, we welcome your pull requests!
  • 7 months ago
  • 2
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

Easily add Census boundaries to your slippy map

Beyond the site itself, one of our goals with Census Reporter is to provide tools that help people work with US Census data.

If you use Leaflet to make “slippy maps,” you can now use Census Reporter’s data to add a layer for Census boundaries. This is still somewhat provisional, but we’re happy to work with the community to make improvements for general usability. If this post reads as gibberish to you, you may be better off waiting until this gets a little more testing and refinement.

If you’re sticking with us, here’s a page which demonstrates the basic idea.

If you use Leaflet, add this to your page:

<script src="http://censusreporter.org/static/js/tilelayer.js"></script>

Then, assuming your map is called map, this is all you need to do to add boundaries of the US States to it:

new CensusReporter.SummaryLevelLayer('040').addTo(map);

I am pretty sure we still need to work out some kinks if you want to control the style or interaction with the layer, but that’s why Git invented pull requests.

If you don’t use Leaflet, you can still do this using this basic format for the GeoJSON tile URL:

'http://embed.censusreporter.org/1.0/geo/tiger2013/tiles/' + summary_level + '/{z}/{x}/{y}.geojson'
where summary_level is one of the three-character strings from the list below.

Also, note: this is likely to behave badly if you try to do all census tracts on a map of the entire US. We have some caching strategies, and we will keep looking for ways to optimize it, but really, it’s better for states and counties than for tracts and ZCTAs.

If you have code suggestions, hit us up on GitHub.. You can also tweet at us @CensusReporter or use our UserVoice support system.

Summary Levels

  • 020: region
  • 030: division
  • 040: state
  • 050: county
  • 060: county subdivision
  • 140: census tract
  • 150: block group
  • 160: place
  • 170: consolidated city
  • 230: Alaska native regional corporation
  • 250: native area
  • 251: tribal subdivision
  • 256: tribal tract
  • 310: metro area (CBSA)
  • 314: metropolitan division
  • 330: combined statistical area
  • 335: combined NECTA
  • 350: NECTA
  • 364: NECTA division
  • 400: urban area
  • 500: congressional district
  • 610: state house (upper)
  • 620: state house (lower)
  • 795: PUMA
  • 860: ZIP code
  • 950: school district (elementary)
  • 960: school district (secondary)
  • 970: school district (unified)
  • 7 months ago
  • 12
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+
'\x3cdiv id=\'tumblr_video_container_85560388409\' class=\'tumblr_video_container\' style=\'width:500px;height:313px;\'\x3e\x3ciframe src=\'https://www.tumblr.com/video/censusreporter/85560388409/500/\' style=\'display:block;background-color:transparent;overflow:hidden\' class=\'embed_iframe tumblr_video_iframe\' scrolling=\'no\' frameBorder=\'0\' data-can-gutter data-can-resize data-width=\'500\' data-height=\'313\' width=\'500\' height=\'313\' allowfullscreen mozallowfullscreen webkitallowfullscreen\x3e\x3c/iframe\x3e\x3c/div\x3e'

A simple tour of Census Reporter, prepared for our entry in the Amazon City on a Cloud Innovation Challenge

  • 1 year ago
  • 1
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

Easier Census Data Downloads

By: Ian Dees

We’re working very hard at Census Reporter to make navigating and exploring Census Bureau’s American Community Survey data easier. Today, we’re launching a new data browser and a feature that lets you download data in your favorite format for further exploration in your own tools.

Start by searching for a location you’re interested in to end up at a Profile page. On the profile page, you can use the table search text box under the map to get more details about that place.

image

Once you’ve picked a table, you’ll end up at our new data view. You can compare the data with larger geographies or add individual geographies of interest to you by typing their name.

image

Just like in the existing comparison builder, you can also add several geographies at once by clicking the “Compare ____ within ____”. For example, you could compare all tracts within a county. This new data view interface will soon replace the comparison builder, but for now, you can get to it from a place profile page.

Once you’ve picked the table and geographies you’re interested in, you can view the data as percentages of the total universe (where applicable) or total numbers with measurements of error.

From the same page, you can download data to your computer in various useful formats. We currently support KML, ESRI Shapefiles, GeoJSON, CSV, and XLSX. The downloaded data will have the geographic data included when you select a spatial format (Shapefile, GeoJSON, KML). You can use this data in your favorite programs to perform further analysis.

For example, the map below was created by downloading the population of all Census tracts in the City of Chicago as a Shapefile, loading it in to QGIS, and applying a simple natural breaks styling to it to build a choropleth.

image

Let us know what you think!

  • 1 year ago
  • 8
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

Easier Access to ACS Data

By: Ian Dees

With the most recent releases of American Community Survey data, the database snapshot we’ve been using for Census Reporter has grown by tens of thousands of rows and gigabytes of data. To alleviate some cost, improve query performance, and make our lives easier, we’ve removed all but the 2012 1-, 3-, and 5-year datasets from the database and the snapshot we publish for everyone to use.

Before removing that data we dumped it into SQL files that you can use to load in to your own database. This means you can use our database schema locally or in a database that’s not in Amazon’s cloud.

Download them here (note that the TIGER data is ~16GB, the 5-years are ~13GB, 3-years are ~1GB, and 1-years are ~500MB):

  • 2014: 1-year
  • 2013: 1-year, 3-year, 5-year
  • 2012: 1-year, 3-year, 5-year, TIGER geodata
  • 2011: 1-year, 3-year, 5-year
  • 2010: 1-year, 3-year, 5-year
  • 2009: 1-year, 3-year
  • 2008: 1-year, 3-year
  • 2007: 1-year, 3-year

Once you download the data you’re interested in, you can use psql to load the data into your database. Learn more about that on the PostgreSQL website.

Note that the loaded data is considerably bigger than the compressed files. Specifically, for 2012, here are the total schema sizes:

  • Tiger 2012: 13 GB
  • ACS 2012/1 year: 2.5 GB
  • ACS 2012/3 year: 4.9 GB
  • ACS 2012/5 year: 162 GB
  • 1 year ago
  • 5
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

ACS 2012 5-year Data Release

By: Ian Dees

image

Census released the ACS 2012 5-year estimates on December 17th and we immediately loaded them into our Census Reporter dataset. After working to compress the dataset, we’re making the updated EBS snapshot available today for you to use with your own Amazon Elastic Compute instances.

The new snapshot ID is snap-f17240e0. The existing post was updated with the new snapshot but otherwise the instructions remain the same.

  • 1 year ago
  • 2
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

Next Steps for Census Reporter

By Sara Schnadt

image

Planning in progress after ONA - with Census Reporter Community Liaison/Designer Sara Schnadt, Front-End Developer/Designer Ryan Pitts, Board Member John Keefe, Project Lead Joe Germuska (and Back-End Developer Ian Dees behind the camera). Clockwise from top left.

After coming together in Atlanta for the Online News Association Conference, our team met to plan out our next round of development in response to feedback from journalists at the conference. An intense day of brainstorming and teasing out of ideas led to a big leap forward in how we are thinking about the interface.

image

Melding of the minds and the computers. Census Reporter team begins day-long planning session after ONA

We’re beginning a complete revision of our “Compare places” feature, which currently asks you to pick a set of census geographies before you even get to see the data inside your chosen table. This was a frustration for even some seasoned data journalists using our site at ONA, and definitely tripped up reporters who weren’t already familiar with the way the census organizes things.

Instead, we’ll rebuild this feature as “Explore a topic,” and offer journalists a lot more guidance in selecting the best data table to answer their questions (more on that later). Once they’ve found the right data, we’ll provide a geography search so they can see what the numbers look like in a place they care about. And at that point, we’ll expose tools to help compare that data across similar places (“how does the median income in Chicago compare with other cities in Illinois?”), or see how the data is distributed within the place itself (“which census tracts in Spokane have the highest poverty rates?”)

This is a fundamental shift in our approach to site architecture, and will manifest in several new interface elements that will be defined in the next couple of months. But there are some key considerations that we have identified right away as well.

The first is that we need a way to choose a place that accounts for the nuances of picking census places. We can do this by combining the core logic of the ‘Use your current location’ view (extending it to work for any geography you choose) with a click-able map that shows you all of the Census geographic boundaries surrounding your selected place.


image

Preliminary interface idea for picking a place (building off of the ‘Find places based on your current location’ tool)

To deal with the daunting task of searching thousands of tables with sometimes obscure-sounding census-ese names, we are also planning to tag and (possibly) annotate all of the tables in the ACS. This will also make the tables themselves easier for journalists to browse, so we are not dealing with names like this:

image

And, we are planning topical landing pages that are as curated as our Profile a Place pages to guide users to finding important topical and related data. We will keep the current Compare Places interface as well so that this more fine-grained tool is available for the census-savvy.

The other big new idea is a global pop-up panel that can appear wherever you are and give you at-a-glance contextual information about the data or visualizations you are looking at, changing the details it displays as you mouse over the page. The exact solution will evolve as we build, but will support the general idea of simultaneously offering the user the most relevant information on a topic or place and detailed contextual information. We looked at WNYC’s Bike Share Stations Map as a good example of this kind of panel in action.

image

Clicking on a paddle makes this panel pop up, displaying related data for that location.

We also came up with solutions for the three pivot-points for place-based data that users requested during our demos at ONA. For the first one, the ability to compare profiles of multiple places, we will create shopping cart functionality for cherry-picking places to see alongside each other, displaying them with comparable ‘apples to apples’ statistics and visualizations.

For the second, time-based pivot point for Profile of a Place, we are drafting a way to see the data over time. This is a bit challenging because there have been changes to the questions asked in the survey and to the boundaries used to tabulate responses. Also, for the smallest geographies (population less than 20,000), technical details about the ACS methodology mean that there is no historic value suitable for comparison.

The third pivot point, the detailed, deeper dive view into topics on the profile page, will be integrated with the new topic landing pages, creating one of several tie-ins between topic and place-based searching.

The other ideas that journalists brought up to us are in the hopper as well, and most will fold into the main solutions outlined here. Others will likely appear in a new dedicated section we are envisioning for more in-depth, analytical, contextual and exploratory features.

While many excellent new ideas that came out of our time at ONA (some from journalists, and some inspired by their experience), our biggest takeaway is that we are basically on the right track. All of the new additions to the Census Reporter site will just make the project stronger and more relevant to journalists’ actual work-flows. We came away from our time in Atlanta newly invigorated and ready to begin the next round of work.

image

As we begin to build new features and improvements for the Census Reporter site, we encourage you to tell is what you think of our plans in our Feedback Forums.


  • 1 year ago
  • 1
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

ACS 2012 3-year Data Release

By: Ian Dees

Census released the ACS 2012 3-year estimates on November 14th and we immediately loaded them into our Census Reporter dataset. Today we’re making the updated EBS snapshot available for you to use with your own Amazon Elastic Compute instances.

The new snapshot ID is snap-0efc2417. The existing post was updated with the new snapshot but otherwise the instructions remain the same.

  • 1 year ago
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

Learning from YOU - Test-driving Census Reporter BETA at ONA

By Sara Schnadt

The Census Reporter team was present in force at the Online News Association conference two weeks ago in Atlanta. We all came out to show off the newly launched beta site and talk to attendees one-on-one about Census Reporter. We were there as part of the Knight Village area at the Midway, which featured some of the most exciting new technology projects in journalism in a hands-on exploratory setting. In addition to demo-ing the site during office hours, doing a lightning talk, and presenting on a civic data panel, we spent most of our days holding down the fort at the Knight Village talking to journalists in-depth about their work, their use of Census data, and watching them try out our new site.


image
Project Lead Joe Germuska talking to Claudia Núñez about Census Reporter at the ONA13 conference

This process was invaluable, and put our work of the past several months into rich perspective. We have approached the site build process iteratively, based on extensive requirements development from interviews with journalists, and developing in the open. And, we have been working intentionally towards ONA as a half-way mark, creating only the core site functionality so that we could take the opportunity to plan the fully-fleshed-out site based on feedback from journalists responding to our work.


image

Community Liaison/Designer Sara Schnadt and Back-End Developer Ian Dees ready to demo the site.

So, having everyone on our team there in person at ONA to hear first-hand how the site is helpful, where the interface might need more work, and what else journalists would like to see there, was very informative to how we are conceiving of and building the site.

To make the most of being together in person (we are usually distributed around the country), Joe Germuska, John Keefe, Ryan Pitts, Ian Dees and met the day after the conference to make plans for the second half our build based on the feedback we had all heard. We were pleased to learn from all the folks we talked to that we are generally on the right track, there are no major features we are missing, and that this tool will make their work genuinely so much easier than it is now. We heard this consistently from shoe-leather reporters, app developers, data journalists, journo nerds, people with small regional beats and people with national beats.

After three days at the Midway at ONA, we had two main takeaways. The first was that the Profile a Place page is great, but we should create three pivot points for it: data over time; this same data for multiple custom places; and more in-depth and interactive data on this place. The second takeaway was that the Compare Places tool needs a new interface.

Other good ideas came out of the demo process as well, some confirming our own plans. These include: make the maps clickable as a way to navigate to data about this place or another place; include AP-style text about each statistic and visualization on a page to make the data more real and relatable (and easy to drop into a story); make all visualizations everywhere embeddable; put small contextual ‘spark line’ style visualizations next to key data points rather than national percentile #’s; always show margin of error on numbers; generate interesting ranked lists on topics and places; make something resembling a google search for the site for both finding a place and a topic; and, recommend interesting data on places that are outliers on a topic for lead generation purposes.

image
As we begin to build new features and improvements for the Census Reporter site, we encourage you to tell is what you think. All the ideas mentioned in this post are included in our new Feedback Forums (where you can comment and vote on them). You can also suggest new features, and give your feedback on the current site.

DETAILS ON WHAT WE LEARNED FROM JOURNALISTS

Profile of a Place View


image

Top area of Census Reporter ‘Profile a Place’ page with data for Los Angeles

Everyone we talked to found our profile page to be the most helpful and user-friendly part of the site so far. It features topical top-level information and visualizations for any geography in the US. We also got some great suggestions for how to make it better. With few exceptions, journalists wanted to be able to compare trends over time for the statistics and visualizations on Profile of a Place. Most journalists also expressed an interest in easily comparing one place to another, either a neighboring place, a larger geography (their city plus the whole state) or another place in the country (e.g. New Yorkers comparing their place to other large metro areas).

image
Another section of ‘Profile a Place’, with data for Los Angeles.

There was also an interesting trend that one insightful journalist pointed out to me - she learned this from her readers. When you live in a smaller place, you are more likely to want to compare your home to neighboring towns and the immediate region. But if you live in a larger place, you set your sights on a larger geographic area (other cities like yours around the country) for contextual statistics for your place. I also learned that readers in smaller places are frequently looking to localize the kinds of statistics they hear about in the national news, to bring them home and understand how they impact their community. Building a very intuitive geographic comparison tool as part of the Profile a Place page will make it possible to get all these kinds of contexts.

Other journalists who are more data-analysis inclined expressed an interest in digging deeper into the topics on the Profile Page, getting more details and more data points on the main topics (Demographics, Economics, Families, Housing, and Social) as well as cross-referencing and sub-dividing the information. For example, one journalist wanted to see how the educational attainment or poverty level visualizations would break down by race.

The number of topics and themes running through census data are quite vast. We curated them for this page by surveying journalists, calculating the topics that have highest frequency in the data, looking at the Census QuickFacts site, and by reviewing the kinds of issues that frequently come up in articles written from census data. While people we spoke to in person were generally happy with this curation, many wanted to see more detail and more data points on each main topic. We would also love to hear your thoughts on this.

Compare Places View

imageCurrent ‘Compare Places’ interface for picking census topical tables

The other main feature we built to try out on journalists at ONA was the Compare Places tool, designed to help journalists identify a topical table from the nearly 1500 tables available in the Census’ American Community Survey. As it is now, Compare Places helps the user filter through all of these tables and narrow down to a browsable quantity to choose from by combining a keyword search and topical filters. In addition to making the process of getting at a specific table for download significantly easier than it is with any existing census tools (we have received some rave reviews so far from app developers - and new apps have already been made), this tool lets you explore the data as well.

You can view the data as a table or as a choropleth map that lets you pick any row in the table and view it geographically. Many journalists I spoke to were excited about this view for its lead generation potential, and spent a lot of time playing with it to see nuanced and contrasting dynamics in various data (such as children in poverty or mobility of immigrant populations) related to their beat or state. Some patterns visible in the data confirmed what they already knew and some were surprising, speaking to the exploratory value of this kind of visualization.


image

A choropleth visualization of a selected topic table in the ‘Compare Places’ tool

There is also a distribution chart view into each table which allows you to see which geographies in your selected area are more and less similar to each other on each data point or detailed topic in the table. Mostly we got a simple ‘wow’, and ‘fascinating’ response for this visualization so far. But it can be quite useful for more in-depth analysis.

While Compare Places works well functionally as it is, and is extremely powerful, pretty much every journalist we showed it to found it difficult to use. This is essentially because its search logic is built to reflect how the data is structured rather than how journalists think about beginning to research a census topic.

Chris Amico, known for his work on Homicidewatch.org, stopped by our table and talked with Ryan and me. He broke it down for us like this: when he wants to write a story or build a visualization about Census Data, he starts with either one of two simple questions - what place do I want to find out about, or what topic? And so our interface should start out as simply as this. He even white-boarded with us:


image

Idea for primary site interface created in conversation with app developer/journalist Chris Amico during site demos at ONA

This conversation confirmed the disconnect we were getting from journalists. When we ‘drove’ for them, we were able to get the answers to some of their more complex questions, but when they tried to navigate this interface themselves, even the more technical journalists would get puzzled and stuck quite quickly. And so we are taking the user-facing piece of this tool back to the drawing board to re-imagine it.

More Ideas

Other good ideas came out of the demo process as well, some confirming our own plans. These include: make the maps clickable as a way to navigate to data about this place or another place; include AP-style text about each statistic and visualization on a page to make the data more real and relatable (and easy to drop into a story); make all visualizations everywhere embeddable; put small contextual ‘spark line’ style visualizations next to key data points rather than national percentile #’s; always show margin of error on numbers; generate interesting ranked lists on topics and places; make something resembling a google search for the site for both finding a place and a topic; and, recommend interesting data on places that are outliers on a topic for lead generation purposes.

Next Steps

Another blog post is coming soon with details on the new build plans we have made. Our next goal for the project is to finish the Profile a Place section by the end of the year, and then to complete all of the main site features by the NICAR conference in February of 2014. So now is the time to give us your thoughts. We are building this site for you!


  • 1 year ago
  • 1
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+

ACS 2012 1-year Data Release

By: Ian Dees

Census released the ACS 2012 1-year estimates on September 19th and we immediately loaded them into our Census Reporter dataset. Today we’re making the updated EBS snapshot available for you to use with your own Amazon Elastic Compute instances.

The new snapshot ID is snap-3bf31238. The existing post was updated with the new snapshot but otherwise the instructions remain the same.

  • 2 years ago
  • 1
  • Permalink
Share

Short URL

TwitterFacebookPinterestGoogle+
Page 1 of 2
← Newer • Older →

Logo

Census Reporter

Census Reporter is a Knight News Challenge-funded project to make it easy for journalists to write stories using US Census data. Expanding upon the volunteer-built Census.ire.org, Census Reporter will simplify finding and using data from the decennial census and the American Community Survey.

Pages

  • Live Site
  • About
  • People
  • Contact

Follow

  • @@CensusReporter on Twitter
  • censusreporter on github

Top

  • RSS
  • Random
  • Archive
  • Ask me anything
  • Mobile
Effector Theme by Pixel Union