GEOG 5223 Final Project (Lessons 7, 8, 9 & 10)

Reconstructing 1920 Charlottesville, Virginia

by the "SketchTools"  -  December 2003

David Beahm, Tom Bell, John Branigan & Tom Wells

Sections:


Figures and Tables:

  Fig.1 - Example Sanborn

  Tables - Attribute Fields

  Fig.2 - Index Map

  Fig.3 - Street Compromise

  Fig.4 - Sanborn Error

  Fig.5 - Georefrence Error

  Fig.6 - Thematic Map

  Fig.7 - Photo

  Fig.8 - 3D

  Sources

The Final Class Project brings together most of the concepts and software techniques learned during previous projects while simulating a realistic proposal / pilot project effort.   A group effort was encouraged so we assembled a volunteer team dubbed the "SketchTools" by the instructor.  All four of us dig dirt professionally but from a different perspective.  The four SketchTools include; an archeologist, a geologist, a geoenvironmental (civil) engineer, and a soil scientist.

Unlike previous class projects, semi-complete "cook book" instructions were not provided.  Instead, only goals and hints were provided to encourage a student lead project specific GIS design and implementation.  Questions were encouraged and the class message board was well utilized.  Weekly submittals were required to make sure that each group remains "on target".

The final project is based on historical maps of Charlottesville, Virginia copyrighted by the Sanborn Map Company in 1920.  According to the lesson background section:

From the late 19th century through the 1990s, the Sanborn Map Company authored what are arguably the most information-rich maps of urban America. The original intent of the maps was to provide a risk assessment tool to the fire insurance industry. They have come to be an important resource in all kinds of research settings, including urban geography, architectural history and preservation, urban planning, ethnic studies, urban archaeology, genealogy, and many others.

Example Sanborn Map (1920 2A)
Figure 1:   Example Sanborn Map (1920_03a)

Because of their intended purpose, the most detailed aspect of the maps are the buildings. Looking at a Sanborn map, one can find information on construction materials, number of stories, entry-ways, ownership, street address, general (residential, commercial, public) as well as specific building type (office, warehouse, garage), and sometimes the date of construction. Other kinds of information that can be gleaned from the maps include street names and widths, land elevations, utility lines, parks and railroad tracks.

The final project premise states that the Albemarle County Historical Society (ACHS) located in Charlottesville Virginia intends to create GIS data layers from 20th century Sanborn maps to aid in land use change studies, genealogy, and visualization of their city. A proposal has been requested for a GIS that includes layers depicting 10-year intervals from 1900 through 1990.  This final class project has our group of four "GIS practitioners" competing for the large contract by designing and implementing a pilot GIS project based on just two Sanborn maps (supplied in four scanned images: 120_02a.jpg &120_02b.jpg, plus 120_03a.jpg & 120_03a.jpg.)  Thirty scanned map images were supplied so no two students were assigned the same image.  Note: According to the map index, the city population was 15,000 in 1920.  According to the US Census, the population had grown to  45,049 in 2000.


The SketchTools were assigned a pilot project sample work area in the heart of downtown Charlottesville.  The digitized area is outlined below on a portion of a Sanborn Index Map.


Figure 2:

Charlottesville, Virginia 1920 Sanborn Index Map Showing the SketchTool's Sample Work Area

 

Project Database Structure

Five Feature Classes and two Coded Domains were created for the relational database structure used to hold  the pilot project's data.  Both line and polygon class features were defined as follows:

  • BUILDINGS - Polygon (includes houses, and out-buildings)

  • BLOCKS - Polygon (city blocks)

  • RAILS - Line (Rail Road Tracks)

  • STREETS - Line (City Streets)

  • MAPS - Polygon (Sanborn Map boundaries)

All of these features are spatially dependent on each other. Similar to the relational database concepts of primary and foreign keys, the feature classes can be linked through their geometry via spatial joins.  

ArcCatalog was utilized to create the GIS "feature classes" (layers) and define their data structure.  The standard feature class fields are not listed below.  (For example, each attribute table includes an ObjectID field that is maintained by ArcView 8.3 as its primary key.)  For project specific data storage needs, the following fields were added to the standard Attribute tables stored in an ArcView 8.3 Personal Geodatabase.  Two coded domains are also utilized as described below.

A limitation in the test pilot project personal database was found concerning the BUILDINGS Attribute Table.  The Buildings table Height field defined as a short integer should have been defined as a floating point number instead.  Some buildings have their height indicated as 1.5 or 2.5 stories on the Sanborn Map which can not be stored in an integer field.  The heights were rounded-up to the nearest integer value.

Note: A few heights are indicated with feet instead of stories on the Sanborn Map.  These heights were converted to stories based on 10 feet equals = 1 story.  (Probably on the low side.)

   
DOMAINS
Two Coded DOMAINS
           
CODE TEXT     CODE TEXT
d_year 0 1900   d_use R residential
1 1910     C commercial
2 1920     P public
3 1930        
4 1940        
5 1950        
6 1960        
7 1970        
8 1980        
9 1990        
   
BUILDINGS
User Defined BUILDINGS Attribute Table Fields
       
Field Name Data Type Length Domain Table Description
build_id long     "Primary Key"
use text 1 d_use Foreign Key to building use domain
height short     Integer representing number of stories
year (map_year) short   d_year Foreign Key to map year domain
address text 30   Street address
   
BLOCKS
User Defined BLOCKS Attribute Table Fields
       
Field Name Data Type Length Domain Table Description
block_id long     "Primary key"
block text 3   Block number from map
old_block text 3   Old number from map (in parentheses)
year (map_year) short   d_year Foreign Key to map year domain
   
RAILS
User Defined RAILS Attribute Table Fields
       
Field Name Data Type Length Domain Table Description
rail_id long     "Primary key"
year (map_year) short   d_year Foreign Key to map year domain
   
STREETS
User Defined STREETS Attribute Table Fields
       
Field Name Data Type Length Domain Table Description
street_id long     "Primary key"
name text 30   Street name
type text 12   Street type
suffix text 10   Street name suffix
left_from_add long      
left_to_add long      
right_from_add long      
right_to_add long      
year (map_year) short   d_year Foreign Key to map year domain
   
MAPS
User Defined MAPS Attribute Table Fields
       
Field Name Data Type Length Domain Table Description
map_id long     "Primary key"
map text 5   Map id
year (map_year) short   d_year Foreign Key to map year domain
   
Tables:

Feature Class Attribute Table User Defined Fields added to store project specific needs. (The year / map_year is an alias / field name combination specified because field names can not be named "year".)


Technical Procedures

A proposed work flow was submitted on the 2nd week of the Final Project.  The workflow listed many of the individual steps in a tutorial fashion.  Only the major steps are outlined here.  Additional detail is included in the next section (QA/QC).

  • Sanborn Maps Georeferenced - Each Sanborn map needs to be located in space and warped to best fit the surviving landmarks that are most visible on the orthorectified photo.

  • Adjust Map Images with an image editor (such as PhotoShop) - The rectified map images can usually benefit from some pixel level manipulations that sharpen edges and improve contrast. 

  • Digitize and Attribute all Five layers / Feature Classes - This is the most work.  Use the Sketch Tool.....

  • Combine Everyone’s Work - Use the ArcMap geoprocessing wizard to "Merge layers together".  Repeat for each feature class.

  • Edge Match - Map objects that span multiple Sanborn maps will require adjustment because of less than perfect alignment.

  • Edge Data Check - Check for missing and duplicate map objects along the Sanborn map boundaries.  (Delete duplicates and add missing map objects.)

  • Merged Data Check - Sort and run queries against the merged attribute tables looking for problems. 

  • Edit Attribute Data - Use the Microsoft Access table column Find/Replace action to make the data more consistent.

  • Generate Attribute ID numbers - VBA code was developed using Microsoft Access to populate the attribute tables.

  • Export the Feature Classes as Shape Files - Use the ArcToolbox.  For possible data use by ArcView 3.x

  • Project Shape Files to GCS_North_American_1983 - ArcView 3.x is limited to a Geographic Coordinate System

 

Error-Checking and Error Correction

Quality assurance/quality control procedures (QA/QC) for the proposed project will follow a plan of data entry, visual inspection, and standard queries to test for errors of entry and database structure.  Data entry tasks include heads up digitizing of lines and polygons and keypunch entry of attributes for the same lines and polygons.   The primary data source for the pilot project is 30 Sanborn map images from 1920.  Data captured consists of street centerlines with address ranges, building footprints with addresses, heights, and use category, rail lines, Sanborn map extents, and municipal block footprints.  Individual and groups of maps will be digitized by more than one technician and fitted together in a final mosaic for each decade.  Attributes will be added by each technician at the time of digitization.

Our pre-bid feasibility study used four technicians who each digitized one map.  Only the street centerline and building footprint tasks were implemented.  QA/QC began at the digitization stage by careful attention to detail by each technician.  Independent error checking began at the merge stage as the individual contributions were assembled into a working prototype GIS.  The following procedures where employed for each task:

STREET CENTERLINES

  1. Add individual street feature classes to the same ArcMap document

  2. Use the ArcMap GeoProcessing Wizard to merge all street feature classes

  3. Start edit session, confirm, merged streets are target, and select Modify Feature as task.

  4. Set snapping to edge and vertex.

  5. Perform 100% check and correct any under or overshoots.

  6. At individual map edges, estimate a compromise junction point and adjust junctions accordingly (see Figure 3).

  7. Delete duplicate segments.

  8. Perform 100% check and confirm segment direction is consistent with address range direction.

  9. Perform 100% check and confirm validity of address ranges for each segment in attribute table.

  10. Save edits and map document then close ArcMap

  11. Open streets attribute table in Access and sort the data various ways to help look for trouble, etc. (see the queries below).

  12. Use Edit>Replace to adjust records so that spelling, capitalization, and punctuation are consistent throughout.

  13. Use VBA script to assign unique Street IDs. This approach was also used to give the Street IDs some meaning.  The streets were sorted by street name and starting address before the sequential numbers were generated & assigned.


Figure 3:

Two screen shots showing: 1.) Mismatched individual street centerlines from adjacent maps, and 2.) Final compromise junction shown in red.


BUILDING POLYGONS 

  1. Add individual building feature classes to the same ArcMap document.

  2. Use the ArcMap GeoProcessing Wizard to merge all building feature classes.

  3. Start edit session, confirm, merged buildings are target, and select Modify Feature as task.

  4. Set snapping to edge and vertex.

  5. Perform 20% check of vertices and polygon alignment with Sanborn map source.  If error rate exceeds one in ten, repeat with an additional 20% check until overall rate is below 10%.

  6. Delete duplicate structures.

  7. Confirm validity of addresses and use for each structure in attribute table.

  8. Perform 20% check of structure heights.  If error rate exceeds one in ten, repeat with an additional 20% check until overall rate is below 10%.

  9. Save edits and map document then close ArcMap

  10. Open building attribute table in Access and sort the data various ways to help look for trouble, etc. (see the queries below).

  11. Use Edit>Replace to adjust records so that spelling, capitalization, and punctuation are consistent throughout.

  12. Use VBA script to create unique Building IDs.  This approach was also used to give the Building IDs some meaning.  The buildings were sorted by address number and street name before the sequential numbers were generated  & assigned.

EXAMPLE ACCESS QUERIES

How inconsistent Street names were spotted:

SELECT Trim(Right([address],Len([address])-InStr([address]," "))) AS [Street Name], Count(buildings.address) AS [Count Street Name]
FROM buildings GROUP BY Trim(Right([address],Len([address])-InStr([address]," ")))
ORDER BY Trim(Right([address],Len([address])-InStr([address]," "))), Count(buildings.address);

How the buildings were sorted before the Building_IDs were assigned:

SELECT Trim(Right([address],Len([address])-InStr([address]," "))) AS [Street Name], Left([address],InStr([address]," ")-1) AS [address number], buildings.address, buildings.build_id
FROM buildings ORDER BY Trim(Right([address],Len([address])-InStr([address]," "))), Left([address],InStr([address]," ")-1);


Database Error Analysis

One source of error is the Sanborn Maps.  We can not go back to 1920 and measure the city but we can spot inconsistencies.  When the first two maps were merged together (1920_02a & 1920_2B) the alignment was disappointing. A better match was expected along 1st Street because both maps share relatively well defined landmarks for georeferencing.

The intersection of Jefferson St. and 1st St. was studied closely.  Both SketchTools georeferenced their map image using the same North-West corner of the city park (the South-East corner of the intersection).  Differences in the georeferencing can be seen in the two screen shots below (with semi-transparent Sanborn maps shown over the modern orthorectified photo).  However, the two Sanborn maps disagree with regard to east-west street alignment and widths.  Map 1920_02a shows straight center-line alignment through the intersection whereas Map 1920_03a shows a noticeable offset with Jefferson St. N.W. significantly wider.



 

Figure 4:

Two screen shots showing the intersection of Jefferson St. and 1st St. - Map 1920_02A versus map 1920_03A


Sanborn map 1920_02A was particularly difficult to georeference because all of the potential landmarks from 1920 in the western half of the map no longer exist.  Georeferencing variability was demonstrated by having a second SketchTool georeference the same map.  The resulting spatial difference is demonstrated in the following figure by the different locations shown for the same buildings (in a few select cases).  However, the four maps fit together well enough to aid in land use change studies, genealogy, and visualization of the city as shown in the figures.  




Figure 5:

Georeferencing Error Illustrated by Comparison. - Two SketchTools georeferenced Map Image 1920_02a.  Difference in the georeferencing are illustrated by offset building locations in a few select cases.  The large building was the McGuffey Primary School in 1920 (now used as an "Arts Center").  The South-East corner is offset by 5.3 meters.

 

Metadata

The Metadata (data about the data) associated with this pilot project was entered in ArcCatalog and exported as HTML.  This metadata is available for viewing via the following links.

 

Thematic Map & Possible Applications

The assigned Sanborn maps included  current usage for the vast majority of the buildings.  In the few cases where information was not included, usage was judged based on nearby buildings.  The SketchTool's assigned sample work area literally includes the center of town as defined by the intersection of 1st Street and Main Street.  The following thematic maps shows that this is a commercial area as one would expect.


Figure 6:

Thematic Map of Charlottesville, Virginia - Building usage classified as shown.


The following figure can be used to judge structural change from 1920 to modern times.  The western portion of the downtown area appears to have undergone urban renewal since 1920.  Metadata was not provided with the supplied orthorectified photo but we are told that it is a Digital Orthophoto Quad (DOQ).  According to Earth Explorer, a Charlottesville East SW 3.75 Minute DOQ dated Saturday 12 March 1994 is the most recent available from the USGS.


Figure 7:

Comparison of Historic and Modern Development in downtown Charlottesville Virginia - An Example of Future Spatial Database Applications possible with this GIS.


3-D Visualization

One of the SketchTools has access to the optional ArcGIS 3D Analyst extension.  This optional ArcView 8.x extension was used to generated the following 3-D rendering based on the Building Height attribute field.  This software was also utilized to generated a "fly-over" 3-D Movie (in "avi" format) from the SketchTool pilot project data set..  Note: the G5223_FP_3D-Movie.avi file has been tested with Windows Media Player version 9.  The 4.13 MB avi file will take a significant amount of time to download even over a fast internet connection. 


Figure 8:

3D Visualization of Charlottesville, Virginia (looking ~ east)  Note: a 3-D Movie is available for download by clicking here.

 

Proposal Estimate

The digitizing process was progressing relatively rapidly as the end of this tedious project task approached.  Based on the pilot project and some anticipating additional speed improvement, the following estimate was made for the time needed to digitize all of the maps for all ten decades.

     
 
Time Estimate per Map Image  
Map Setup & File Maintenance 0.5 hour
Georeference Sanborn Image 0.5 hour
Digitize Street feature class 0.75 hour
Digitize Building feature class 1.25 hours
Digitize Blocks feature class 0.5 hour
Digitize Maps feature class 0.25 hour
Total per Map Image:  3.75 hours
   
Project Total Estimated Time  
* 30 Maps per Decade    112.5 hours
* 10 Decades (1900-1990) 1,125.0 hours
Note:  The digitization estimate assumes that each decade includes a similar set of 30 map images.  Some additional time should be added to include digitizing the Rail Road layer. 

Additional project expenses could include the design of a web-site that utilizes the proposed GIS.  At say, $60/hour, the digitization process alone would cost $67,500.

 

 



Sources


GEOG 5223 CD-R, WCGIS\Lesson7_8_9_10\ Datasets

GEOG 5223 On-Line "In Touch" class discussions, December 2003

United States Geological Survey Status Graphics http://mapping.usgs.gov/www/products/status.html Accessed 15 December 2003.

United States Geological Survey (2002) Geographic Names Information System. http://geonames.usgs.gov/ Accessed 17 December 2003.

Go To the Top of Page