Reference at Newman Library

Tag: New York City

New Tutorials for Census Data

My team in the GIS Lab has just finished writing two new tutorials to help students and faculty find neighborhood census data. The tutorials are in a PDF format and can be used either for hands-on exercises in class or as handouts for self-directed learning.

1. Anastasia has written a tutorial on using the City’s NYC Census Factfinder. She demonstrates how to look up current census profiles for neighborhood tabulation areas and census tracts, and how to combine census tracts to create profiles for user-generated neighborhoods.

2. Janine has written a tutorial for the Social Explorer database. She demonstrates how to navigate the interface, make good looking maps, and how to download data tables for census geographies or user-generated geographies (also built by using census tracts).

These tutorials as well as others previously mentioned (on finding NYC Census data and on using the American Factfinder) are hosted in two places. I’ve created individual boxes for each one on a new Tutorials tab in the NYC Data Guide here: http://guides.newman.baruch.cuny.edu/nyc_data/tutorials. Since each tutorial is in its own box, you can easily embed the ones you want in your own guides. The PDF files themselves are stored on the Baruch Geoportal’s server, and can also be accessed centrally from there.

Updates to the NYC Neighborhood Handout

For those of you who visit courses and do presentations on finding neighborhood data, I’ve just updated my handout “Finding NYC Neighborhood Census Data“. It summarizes the types of geographies, datasets, and resources that are available for finding neighborhood-level census data. The primary change was that I added a new resource called the NYC Factfinder; this is a web-map application produced by the Dept of City Planning that lets users get basic census profiles for census tracts and neighborhood tabulation areas (NTAs). They’ve added a new feature where you can also build your own neighborhood profiles by selecting census tracts. For additional info about the NYC Factfinder you can read this post.

The handout is embedded in key places in the Lib Guides – in the NYC Data (on the Neighborhoods tab) and in the US Census guide. I also updated my American Factfinder tutorial last spring, and it’s also embedded in these guides. Each tutorial has its own box, so if you wanted to embed either one feel free (the boxes are named Finding NYC Neighborhood Census Data and American Factfinder respectively). It’s better to link rather than copy, as I update the tutorials every year or so.

My team is currently working on two new tutorials that are in a similar vein to the American Factfinder tutorial: one for the new NYC Factfinder and another for the Social Explorer. We should have them ready in about a month. I’ll be updating the NYC neighborhood handout again in the spring, once the latest American Community Survey data for 2014 is released at the end of this year. The Census Bureau is dropping the 3-year ACS estimates with the 2014 release, so we’ll just have a choice between 1-year and 5-year estimates. At that point I’ll remove any reference to the 3-year numbers.

NYC Mass Transit Spatial Layers for GIS

The GIS Lab has just launched a new dataset called the NYC Mass Transit Spatial Layers series. These GIS files contain the stops and routes for NYC buses, subways, and trains (the Long Island Rail Road and Metro-North). The files are released under Creative Commons and are available on the Baruch Geoportal at: http://www.baruch.cuny.edu/geoportal/data/nyc_transit/.

We intend to update the data bi-annually. The layers are provided in a shapefile format that can be used with just about any GIS software, and are projected in the local state plane coordinate system commonly used by City agencies. In the future we plan to add an FAQ to provide new users with tips on how to work with the files.

The series was created so that members of the public would have access to well-documented and readily-usable GIS layers of NYC mass transit features. The MTA publishes a data feed for developers that contains data for stops, trips, and routes, but in a format that’s not readily useable in GIS (for example, routes have to be constructed from long strings of coordinates) or readily useable for making basic map representations of routes or stops (the data is in a General Transit Feed Specification format that’s suitable for constructing schedules and timetables). We have essentially taken this raw data, built a process for transforming and cleaning it, and have documented it using spatial metadata standards.

The process was a team effort that CA’s Rachel Weinberg, Josiah Wahlrab, Araby Smyth and I all contributed to. The CUNY Mapping Service at the Center for Urban Research also deserves recognition – they inspired this project through the detailed documentation they provided on their experiences working with this data feed.

lirr_routes

Accessibility to Jobs for NYC Neighborhoods

The Rudin Center for Transportation at NYU recently released a report entitled Mobility, Economic Opportunity and New York City Neighborhoods. Included with the report are an interactive map and summary profiles by ZIP Code (together in one PDF) that illustrate how accessible each neighborhood is relative to employment opportunities. In addition to illustrating accessibility, the data also seeks to illustrate the balance or imbalance between available job opportunities (based on the type / sector of work) versus the education, skill, or income level of different neighborhoods.

Accessibility measures and transit routing were generated using the Google Apps API, and the other data comes from the Census: neighborhood demographics from the 2008-2012 American Community Survey, and job counts from the Longitudinal Employer-Household Dynamics (LEHD) Origin-Destination Employment Statistics. This would be a good resource for our students who are doing neighborhood research.

rudin_jobaccess_nyc

NYC Data Guide Neighborhood Maps using CartoDB

The city maps on the NYC Data neighborhood page allow users to view neighborhood areas and click on them to view data directly from the US Census Bureau. There is a map for PUMAs (census statistical areas that have 100k people and are similar in size and shape to community districts) and a map for ZCTAs (ZIP Code Tabulation Areas, which represent USPS ZIP Codes that have a geographic area). The PUMA maps point to the latest 3-year profiles from the American Community Survey (ACS), and the ZCTA maps point to profiles from the 2010 Census and the latest 5-year ACS dataset.

Both maps have just been updated to point to 2013 data that was recently released. They can be viewed within the guide, but can be navigated more easily if you hit the full screen button that’s embedded in the map.

Previously, the maps were overlaid on a Google Maps basemap. Unfortunately Google decided to deprecate the ability for people to overlay KML files stored on webservers over their maps. The alternatives they offer are completely unsatisfactory. So, we are now using a free service from CartoDB, a web-mapping service headquartered right here in the Flatiron district, as our new platform. Many thanks go to Data Services CA Rachel Weinberg for researching and experimenting with CartoDB so we could keep this service up and running!

puma_maps2013

NYCdata Update – NYC Government Structure

As a result of our meeting back in December, the NYCdata team at the Weissman Center has updated their resource based on feedback we provided. They’ve added a sub-section under the Public Safety and NYC Governmental Structure chapter that provides links to government agencies within each of the three branches of government. Over the next six months we will be working with them to update the maps on their site.

New Census ACS Data and Maps

The Census Bureau finished its roll-out of the latest American Community Survey (ACS) estimates in December. I’ve updated the neighborhoods tab of the NYC Data guide to point to the latest summary profiles:

  • Google map of NYC ZCTAs (ZIP Code Tabulation Areas) points to the latest five-year estimates: 2008-2012
  • Google map of NYC PUMAs (statistical areas with 100k people that approximate groups of neighborhoods) points to the latest three-year estimates: 2010-2012
  • Links for the city as a whole and each individual borough / county point to the latest one-year estimates: 2012

The PUMA-level data for the 2012 ACS uses new PUMA boundaries that were redrawn for the 2010 Census; previous editions of the ACS (from 2011 back) used 2000 Census boundaries. While the boundaries are only slightly different, the biggest change is that the Census has assigned names to the PUMAs based on local government input; previously the PUMAs were numbered but not officially named. The Google Map has been updated to reflect this change, and Joe Paccione has created a series of static PDF maps (avaliable on the same page under the link “PUMA Maps”) that depict the PUMAs with their numbers and new names.

NYCdata Presentation Dec 9th

NYCdata (http://www.baruch.cuny.edu/nycdata/) is a public web-based compendium of statistics about the City of New York. Created by the Weissman Center for International Business, this comprehensive resource is organized into categories of statistics (population, business, finance, trade, culture, etc.) and each table references its underlying sources. The data is updated on an on-going basis.

Eugene Sherman, a Fellow at the Weissman Center, and Project Manager Kenya Williams are going to give a 30 minute presentation to the library on Monday Dec 9th at 10am in room 320a. The website recently went through a re-design and they would like to illustrate the updates and generally promote the resource. All are welcome to attend.

MapPluto Tax Lot / Parcel GIS Data

Over the summer the NYC Dept of Planning changed course and started providing their MapPluto product – a GIS dataset with boundaries of every tax parcel and detailed attributes like zoning, land use, land value, building descriptors, and administrative identifiers – for free. Previously the dataset cost $300 for each borough and had very tight restrictions on use. Now anyone can download the latest version from their website.

We had purchased a copy of the 2008 MapPluto data for all five boroughs, and I handled the few requests I had via email. But now that the restrictions are off I’m providing free access to it on the Baruch Geoportal. The data is in shapefile format and can be used in any GIS system (ArcGIS, QGIS, etc.). There is one file for each borough. At this point it’s useful for historic purposes; users who want the latest data should go directly to the City’s website.

IRS Tax-Exempt Organizations in NYC

The IRS Office of Statistics on Income publishes a dataset that lists the names and addresses of all tax-exempt organizations in the country that are required to file with the IRS: the Exempt Organizations Business Master File Extract.

Since we get many questions about non-profits in the city, and since this resource is raw data that isn’t readily useable for all purposes (selecting just the records in NYC is actually quite a chore), I’ve created a subset that contains just the records for the five boroughs. The data is available in a spreadsheet file that contains: one metadata sheet, one sheet that lists all of the organizations, and one that summarizes them by borough and exempt organization subcode. The large majority of records in the file are classified as 501(3)c organizations, which include most public charities and private foundations.

I’ve modified the records by adding a ZIP-5 code, a county/borough code, and by cleaning up and standardizing the city name in the address field; otherwise the records are exact duplicates of what appears in the original IRS file. The records represent all tax-exempt organizations that filed a 990 Form with the IRS – it doesn’t represent all non-profits. Religious organizations, state and federal public institutions, and small charities with annual revenue less than $50,000 are not required to file (but some do anyway). Since the IRS extracted this information directly from forms submitted by filers, records may contain spelling or classification errors and could appear in duplicate. Users will need to bear this in mind, and may have to clean the data further based on their intended purpose.

The file is available via the NYC Data LibGuide, on the Health and Human Services tab in a dedicated box I’ve created called IRS Tax-Exempt Organizations in NYC. If you have a LibGuide for a course or another subject feel free to *link* to this box in your guide if you find it relevant. I’ve created a process that enables me to easily update the data, which I plan on doing on a quarterly basis (the IRS updates its master file monthly). If you link to the box the contents will be automatically updated as I release new versions (whereas if you create a copy, the file will become stale as the copy is severed from the original box).

For more information and full documentation about the dataset you can visit the IRS Exempt Organization Master File page (I also provide a link to it from the LibGuide box).