Was curious if anyone else has worked with the bulk data downloads of survey plats, control document index and patent records available at GLO Records Web Services and Bulk Data Web Page ? The raw data is provided in multiple "CSV" text files that are updated each month. The text files are comma delimited and the data type of all the fields is character (each field value bracketed with double quotes).
Here is the link to the download page . The patent records are available by individual state. The control document index, land status and survey bulk data downloads contain data for all the states with records on the GLO Records web site. With a bit of effort I've been able to figure out the data structures of the various text files and import them into databases. While the Bulk Data web page indicates that information on the data structures can be obtained via email request, the documentation remains to be completed.
I'm trying to figure out the relational nature of the data so I can eventually link the survey plats and field notes to the control document index and patent records. Using the unique code assigned to each plat or document one can use the GLO Records' "Web Services" feature to print a data dump of that plat or document. I can provide some samples of the data if anyone is interested.
I've used the bulk downloader from the National Map (USGS) but not for GLO Records yet. Are the coordinates the same or similar as those from GCDB? If this is something available in CO, based upon the availability of other records for CA, it won't be available for the records here until about 2024.
I've never been able to access notes from that site, I just get the BLM office to send them in one state, in another state the BLM has their own site
Evan,
The bulk data contains information regarding the plats, field notes and other documents available on the GLO Records web site. It is not like the GCDB so no coordinates other than township and range.
Here is a description of the data available for survey plats and field notes. The file "All_Survey.zip" contains survey plats and field notes for 37 states. The zip file contains 26 files, several of which are "lookup" files. Here is the list of data files: County.csv, Document_Action.csv, Document_Name.csv, Document_Management.csv, FN_Image.csv, FN_Image_Link.csv, FN_Part.csv, FN_Volume.csv, Instruction.csv, Plat.csv, Plat_Image.csv, Plat_Link.csv, Plat_To_Plat_Link.csv, Survey.csv, Survey_Instruction_Link.csv and Township.csv. The lookup files (usually comprised of coded lists of standardized information) are: Action_Type_Lookup.csv, County_Code_Lookup.csv, Duplicate_Status_Lookup.csv, Instruction_Type_Lookup.csv, Meridian_Lookup.csv, Mining_District.csv, Name_Type_Lookup.csv, State_Code_Lookup.csv, Survey_Office_Lookup.csv and Survey_Type_Lookup.csv.
The first line of each csv file contains the field names. In order to assemble all of the data associated with a particular survey, you would first start with an ID number unique to that survey (usually called "DM_ID"). The Survey.csv database would be searched on the DM_ID, followed by the Plat files, Township.csv, County.csv, Instruct.csv, Document_Action.csv and Document_Management.csv files. If there are field notes, the field notes files would then be searched. The Plat_Image.csv and FN_Image.csv files contain the links to the view the plat and field notes.
Below is a data dump for a mineral survey that has both a lode and mill site. Because of this there are two DM_ID numbers. Along with the bulk data, the GLO Records web site also has a web services function that will provide a data dump for a given DM_ID number. Here is the LINK to the web service "getSurveyDetails". If you enter "220233" and "220234" for the DM_ID you will get two data dumps that look pretty meaningless. If you save the page as an "XML" file and then open the file with an XML editor you will get a good view of how all the data is organized and related to each other along with the field names for each data element. If you prefer, I can email a couple of example XML files.
Now this isn't something that the casual GLO Records user might undertake. The GLO Records search function is very basic and sometimes you cannot find records. For example, if one conducts a search on mineral survey number, the search function will not work for a mineral survey with a lode and mill site because the search function does not accept characters, only numbers. mineral surveys that have a lode and mill site have "A & B" appended to the survey number. For the example above, the mineral survey is 2176 A & B. A search on the number fails to find the survey plat.
I created a database of all the mineral survey plats in Colorado with hot links to the 25000 mineral survey plats currently hosted on GLO Records, which is one reason I am "playing" with the bulk data. I'm also interested in relating the original mineral survey to later dependent resurveys that include the mineral survey. I'd eventually like to be able to link the survey plat and field notes databases to the patent records and control document index databases so I can pull all the records without visiting or calling the BLM. An added bonus is that the bulk data also includes mineral surveys in Montana, Idaho, Oregon, Arizona and New Mexico (there are only 10 or so California mineral survey plats online).
Gene,
I misunderstood your use of the term "raw data". Your explanation clears that up. I'll take a look at the bulk data page you linked in your 1st post a bit later today so that I can start tracking your discussion better. I'll likely have some questions after I get into the info available.
Thanks for putting this on the forum.