Web Service Help
Introduction
Publications Warehouse uses a number of web services in order to generate its dynamic content, by querying our database of USGS Publications. Some of these web services may be helpful to outside developers for many purposes, and they can be used by anyone who is interested, as an alternate way to access the data available in Pubs Warehouse. This document details how you can make requests to our web services, and receive the results in a format that is convenient for your needs.
The Pubs Warehouse web services may be queried using a RESTlike (REpresentational State Transfer) technique. This method will retrieve the same data as the WQP form when the same retrieval parameters are specified. The basic building blocks for querying the services are outlined in the following series of tables. Most non-alphanumeric characters (such as punctuation) must be "url-encoded", (for example: space is "%20").
Parameters must be appended to the base Uniform Resource Locator (URL) for the RESTlike Pubs Warehouse web services. The base URL is constructed differently depending on the type of information being requested. There are two options - one base URL for downloading sites and one base URL for downloading results:
- Base URL for downloading publication information -- http://pubs.er.usgs.gov/pubs-services/publication?
Construct a RESTlike web service query by concatenating the base URL with the desired parameters and arguments (Table 1). At least one parameter-argument pair must be specified. Separate multiple parameter-argument pairs with an ampersand ("&"). Unneeded web service parameters may be omitted. If no mimetype is specified, the retrieval will default to JSON format, which is documented below
Table 1. URL-encoded retrieval parameters and arguments for WQP web services [parameter names are insensitive only to the leading capital letter].
| REST parameter | Argument | Domain Value URL | Discussion |
|---|---|---|---|
| q | any text string | The core pubs warehouse search parameter, searches a text index of all fields of pubs warehouse. Common words (a, as, the, etc) are dropped from the search, and plurals (e.g. hurricane vs. hurricanes) are combined. | |
| title | text string | An exact match for the string within the title of a publication | |
| contributingOffice | text string | https://pubs.er.usgs.gov/pubs-services/lookup/costcenters?mimetype=json | The name of the USGS organization which contributed this publication. There is a domain value service for contributing office. The data behind this particular field is best starting in roughly 2012, though some offices have much better data. |
| contributor | text string | any text string matching a contributor, with the right side if the sting wildcarded. For example, wild will match both wild and wildlife. For more recent publications, the email address of usgs contributors is also indexed. | |
| year | number (4 digit) | exact match for year published | |
| startYear | number | Return publications that were published in or after this year. | |
| endYear | number | Return publications that were published in or before this year | |
| typeName | https://pubs.er.usgs.gov/pubs-services/lookup/publicationtypes?mimetype=json | ||
| subtypeName | https://pubs.er.usgs.gov/pubs-services/lookup/publicationsubtypes?mimetype=json&publicationtypeid=4 | ||
| seriesName | https://pubs.er.usgs.gov/pubs-services/lookup/publicationseries?active=n&mimetype=json&publicationsubtypeid=12&text=as | ||
reportNumber | string | matches a USGS or other agency report number | |
| g | Well Known text polygon | See http://www.opensearch.org/Specifications/OpenSearch/Extensions/Geo/1.0/Draft_2 | |
| page_size | number | number of publications records to return per response, for performance reasons, keeping the page size under 5000 is recommended | |
| page_number | number | page number for paginated results. | |
| mod_x_days | number | returns publications that have been modified within the given number of days. mod_x_days=90 would return publications that were modified in the last 90 days | |
| pub_x_days | number | returns publications that have been made public on the publications warehouse within the given number of days. pub_x_days=90 would return publications that were added in the last 90 days. This parameter may not catch new historical publications, which are often backdated. | |
| mimeType | json | the native format of publications warehouse. While it resembles csl-json, the data model is richer. | |
| xlsx | Output format is xlsx compatible with MS-Excel 2007 and greater | ||
| csv | Output format is comma-separated columns | ||
| tsv | Output format is tab-separated columns | ||
Using the Web Service
The easiest way to sort out using the Pubs Warehouse web service is to use the search interface, which also takes advantage of the API. After building a search, the link "JSON version of this page of results" will provide a web service URL.
- The search for all publications that are linked to the Wisconsin Water Science Center can be seen here: http://pubs.er.usgs.gov/search?q=&title=&contributingOffice=Wisconsin+Water+Science+Center&contributor=&year=&startYear=&endYear=&typeName=&subtypeName=&seriesName=&reportNumber=&g=&advanced=True
- The API call to get the same data in JSON (from the JSON version of this page of results link) is here: https://pubs.er.usgs.gov/pubs-services/publication/?page_size=25&page_number=1&contributingOffice=Wisconsin%20Water%20Science%20Center
- If you prefer a delimited version, you can download TSV, CSV or Excel directly, no paging required:
- Example links: https://pubs.er.usgs.gov/pubs-services/publication/?mimeType=xlsx&contributingOffice=Wisconsin%20Water%20Science%20Center
- TSV, CSV, and Excel versions of the data have a less rich data model, due to the fact that delimited data formats cannot represent hierarchical data well.
- Links are not available, and authors are a single list
RSS Feeds
The base Publications Warehouse RSS feed is https://pubs.er.usgs.gov/pubs-services/publication/rss/
The Pubs Warehouse RSS feed is fully customizable as well, and uses the same parameters as the core web services. For example, to get a feed that returns publications from the Wisconsin Water Science center for the past 60 days,
Data Structure
Here is an example response: https://pubs.er.usgs.gov/pubs-services/publication/?page_size=1&page_number=1&contributingOffice=Wisconsin%20Water%20Science%20Center&typeName=Report
{
"pageSize": "1",
"pageRowStart": "0",
"pageNumber": "1",
"recordCount": 99,
"records": [
{
"id": 70150299,
"text": "ofr20151076 - 2015 - Oil-particle interactions and submergence from crude oil spills in marine and freshwater environments: review of the science and future research needs",
"interactions": [],
"lastModifiedDate": "2015-06-24T12:45:52.000",
"indexId": "ofr20151076",
"displayToPublicDate": "2015-06-24T13:30:00.000",
"publicationYear": "2015",
"publicationType": {
"id": 18,
"text": "Report"
},
"publicationSubtype": {
"id": 5,
"text": "USGS Numbered Series"
},
"seriesTitle": {
"id": 330,
"text": "Open-File Report",
"onlineIssn": "2331-1258",
"printIssn": "0196-1497"
},
"seriesNumber": "2015-1076",
"title": "Oil-particle interactions and submergence from crude oil spills in marine and freshwater environments: review of the science and future research needs",
"docAbstract": "<p>Oil-particle interactions and oil submergence are of much interest to oil spill responders and scientists, especially as transportation of light and heavy crude oils increases in North America’s coastal marine and freshwater environments. This report contains an up-to-date review of the state of the science for oil-particle aggregates (OPAs), in terms of their formation and stability which may alter the transport, fate, and toxicity of the residual oil and, hence, its level of ecological risk. Operational considerations—detection, containment, and recovery—are discussed.</p>\n<p>Although much is known about oil-particle interactions in coastal marine environments, there remains a need for additional science on methods to detect and quantify the presence of OPAs and to understand their effects on containment and recovery of oil spilled under various temperature regimes and in different aquatic habitats including freshwater environments.</p>",
"language": "English",
"publisher": "U.S. Geological Survey",
"publisherLocation": "Reston, VA",
"doi": "10.3133/ofr20151076",
"productDescription": "v, 35 p.",
"numberOfPages": "44",
"onlineOnly": "Y",
"additionalOnlineFiles": "N",
"ipdsId": "IP-059436",
"costCenters": [
{
"id": 677,
"text": "Wisconsin Water Science Center"
}
],
"links": [
{
"id": 302288,
"rank": 2,
"type": {
"id": 11,
"text": "Document"
},
"url": "http://pubs.usgs.gov/of/2015/1076/pdf/ofr2015-1076.pdf",
"text": "Report",
"size": "1.49 MB",
"linkFileType": {
"id": 1,
"text": "pdf"
},
"description": "Report"
},
{
"id": 302287,
"rank": 1,
"type": {
"id": 15,
"text": "Index Page"
},
"url": "http://pubs.usgs.gov/of/2015/1076/"
},
{
"id": 302289,
"type": {
"id": 24,
"text": "Thumbnail"
},
"url": "http://pubs.er.usgs.gov/thumbnails/ofr20151076.jpg"
}
],
"publishingServiceCenter": {
"id": 6,
"text": "Columbus PSC"
},
"contributors": {
"authors": [
{
"text": "Fitzpatrick, Faith A. fafitzpa@usgs.gov",
"contributorId": 127794,
"corporation": false,
"usgs": true,
"family": "Fitzpatrick",
"given": "Faith A.",
"email": "fafitzpa@usgs.gov",
"affiliation": {
"id": 677,
"text": "Wisconsin Water Science Center"
},
"id": 556667,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 1
},
{
"text": "Boufadel, Michael C.",
"contributorId": 143663,
"corporation": false,
"usgs": false,
"family": "Boufadel",
"given": "Michael C.",
"affiliation": {
"id": 15288,
"text": "Center for Natural Resources Development and Protection, Department of Civil and Environmental Engineering, the New Jersey Institute of Technology, Newark NJ"
},
"id": 556668,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 2
},
{
"text": "Johnson, Rex",
"contributorId": 104374,
"corporation": false,
"usgs": true,
"family": "Johnson",
"given": "Rex",
"id": 556670,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 3
},
{
"text": "Lee, Kenneth W.",
"contributorId": 16099,
"corporation": false,
"usgs": true,
"family": "Lee",
"given": "Kenneth W.",
"id": 556669,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 4
},
{
"text": "Graan, Thomas P.",
"contributorId": 97021,
"corporation": false,
"usgs": true,
"family": "Graan",
"given": "Thomas P.",
"id": 556679,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 5
},
{
"text": "Bejarano, Adriana C.",
"contributorId": 143667,
"corporation": false,
"usgs": false,
"family": "Bejarano",
"given": "Adriana C.",
"affiliation": {
"id": 15292,
"text": "Research Planning, Inc."
},
"id": 556677,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 6
},
{
"text": "Zhu, Zhenduo",
"contributorId": 83828,
"corporation": false,
"usgs": true,
"family": "Zhu",
"given": "Zhenduo",
"id": 556672,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 7
},
{
"text": "Waterman, David",
"contributorId": 143664,
"corporation": false,
"usgs": false,
"family": "Waterman",
"given": "David",
"affiliation": {
"id": 15289,
"text": "University of Illinois, Ven Te Chow Hydrosystems Laboratory"
},
"id": 556671,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 8
},
{
"text": "Capone, Daniel M.",
"contributorId": 64167,
"corporation": false,
"usgs": true,
"family": "Capone",
"given": "Daniel M.",
"id": 556678,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 9
},
{
"text": "Hayter, Earl",
"contributorId": 143665,
"corporation": false,
"usgs": false,
"family": "Hayter",
"given": "Earl",
"affiliation": {
"id": 15290,
"text": "USACE, Coastal and Hydraulic Laboratory"
},
"id": 556673,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 10
},
{
"text": "Hamilton, Stephen K.",
"contributorId": 143690,
"corporation": false,
"usgs": false,
"family": "Hamilton",
"given": "Stephen K.",
"id": 556799,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 11
},
{
"text": "Dekker, Timothy",
"contributorId": 143666,
"corporation": false,
"usgs": false,
"family": "Dekker",
"given": "Timothy",
"affiliation": {
"id": 15291,
"text": "Limno Tech, Inc."
},
"id": 556676,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 12
},
{
"text": "Garcia, Marcelo H.",
"contributorId": 74236,
"corporation": false,
"usgs": false,
"family": "Garcia",
"given": "Marcelo H.",
"id": 556675,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 13
},
{
"text": "Hassan, Jacob S.",
"contributorId": 143668,
"corporation": false,
"usgs": false,
"family": "Hassan",
"given": "Jacob S.",
"affiliation": {
"id": 15293,
"text": "USEPA Region V"
},
"id": 556680,
"contributorType": {
"id": 1,
"text": "Authors"
},
"rank": 14
}
]
}
}
]
}
Deprecated services
The Publications Warehouse supports a number of legacy services to support data access by older tools. If you are using these older tools and have questions about using these services, please contact pubs_tech_group@usgs.gov.