OFFICE
OF MANAGEMENT AND BUDGET
Guidelines
for Ensuring and Maximizing the Quality, Objectivity, Utility, and
Integrity of Information Disseminated by Federal Agencies
AGENCY:
Office of Management and Budget, Executive Office of the President.
ACTION:
Final Guidelines, with Request for Comments
SUMMARY:
These guidelines implement section 515 of the Treasury and General
Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554).
Section 515 directs the Office of Management and Budget (OMB) to
issue government-wide guidelines that "provide policy and procedural
guidance to Federal agencies for ensuring and maximizing the quality,
objectivity, utility, and integrity of information (including statistical
information) disseminated by Federal agencies." Within one year
after OMB issues these guidelines, agencies must issue their own
implementing guidelines that include "administrative mechanisms
allowing affected persons to seek and obtain correction of information
maintained and disseminated by the agency" that does not comply
with the OMB guidelines. OMB is also requesting additional comment
for 30 days on the "capable of being substantially reproduced" standard
(paragraphs V.3.B, V.9, and V.10) which is issued on an interim
final basis.
DATES:
Effective Date: October 1, 2001.
Comment
Date: Comments on the "capable of being substantially reproduced"
standard in paragraphs V.3.B, V.9, and V.10 must be submitted by
October 28, 2001.
ADDRESSES:
Please submitted comments to Brooke J. Dickson of the Office of
Information and Regulatory Affairs, Office of Management and Budget,
Washington, DC 20503. Comments can also be e-mailed to informationquality@omb.eop.gov.
FOR
FURTHER INFORMATION CONTACT: Brooke J. Dickson, Office of Information
and Regulatory Affairs, Office of Management and Budget, Washington,
DC 20503. Telephone (202) 395-3785.
SUPPLEMENTARY
INFORMATION: In section 515(a) of the Treasury and General Government
Appropriations Act for Fiscal Year 2001 (Public Law 106-554; H.R.
5658), Congress directed the Office of Management and Budget (OMB)
to issue, by September 30, 2001, government-wide guidelines that
"provide policy and procedural guidance to Federal agencies for
ensuring and maximizing the quality, objectivity, utility, and integrity
of information (including statistical information) disseminated
by Federal agencies... ." Section 515(b) goes on to state that the
OMB guidelines shall:
"(1) apply
to the sharing by Federal agencies of, and access to, information
disseminated by Federal agencies; and
"(2) require
that each Federal agency to which the guidelines apply --
"(A)
issue guidelines ensuring and maximizing the quality, objectivity,
utility, and integrity of information (including statistical information)
disseminated by the agency, by not later than 1 year after the date
of issuance of the guidelines under subsection (a);
"(B)
establish administrative mechanisms allowing affected persons to
seek and obtain correction of information maintained and disseminated
by the agency that does not comply with the guidelines issued under
subsection (a); and
"(C)
report periodically to the Director --
"(i)
the number and nature of complaints received by the agency regarding
the accuracy of information disseminated by the agency; and
"(ii)
how such complaints were handled by the agency."
These
guidelines are to be issued "under sections 3504(d)(1) and 3516"
of the Paperwork Reduction Act of 1995; pursuant to section 3503
of that Act, the authorities of the OMB Director are carried out
by the Administrator of the Office of Information and Regulatory
Affairs.
Background
The
focus of section 515 is on the Federal Government's information
dissemination activities. Indeed, Federal agencies have disseminated
information to the public for decades. Until recently, agencies
have disseminated information principally by making paper copies
of documents available to the public. In recent years, however,
Federal information dissemination has grown due to the advent of
the Internet, which has ushered in a revolution in communications.
The Internet has enabled Federal agencies to disseminate an ever-increasing
amount of information. Congress has strongly encouraged the Executive
Branch's dissemination efforts in statutes that include particular
dissemination activities and in the government-wide dissemination
provisions of the Paperwork Reduction Act of 1995 (44 U.S.C. chapter
35) (the PRA). In addition, the Executive Branch's strong support
for information dissemination is reflected in the dissemination
provisions of OMB Circular A-130, "Management of Federal Information
Resources," as well as in the provisions in OMB Circular A-110,
"Uniform Administrative Requirements for Grants and Agreements With
Institutions of Higher Education, Hospitals, and Other Non-Profit
Organizations," related to a Freedom of Information Act request
for research data relating to published research findings produced
under an award that were used by the Federal Government in developing
an agency action that has the force and effect of law (64 FR 54926;
October 8, 1999).
Section
515 builds upon the existing agency responsibility to ensure information
quality. According to the PRA, agency Chief Information Officers
(CIOs) must manage information resources to "improve the integrity,
quality, and utility of information to all users within and outside
the agency, including capabilities for ensuring dissemination of
public information, public access to government information, and
protections for privacy and security." Before an agency collects
information from 10 or more persons, the agency must seek public
comment "to enhance the quality, utility, and clarity of the information
to be collected." The agency then must obtain OMB approval that
is based upon an evaluation of the agency's need for the information,
the "practical utility" of the information to be collected, and
the minimization of burden that would be imposed on the public in
responding to the collection. The CIO must certify to OMB that the
agency, "to the maximum extent practicable, uses information technology
to reduce burden and improve data quality."
In
developing these guidelines to implement section 515, OMB recognized
that Federal agencies disseminate many types of information in many
different ways. A few examples can only begin to describe the breadth
of information disseminated by the Federal government. Agencies
disseminate statistical information, such as the aggregated information
from the 2000 Census and the monthly and quarterly economic reports
issued by the Bureau of Economic Analysis and the Bureau of Labor
Statistics. Agencies disseminate information that aids members of
the public in their daily activities, such as the National Weather
Service's weather reports and the FAA's air travel advisories. Agencies
disseminate information about health, safety, and environmental
risks and information that they collect from regulated entities,
such as EPA's dissemination of Toxic Release Inventory information.
Agencies also disseminate technical information that they create
or obtain in the course of developing regulations, often involving
scientific, engineering, and economic analysis. Agencies disseminate
information when they issue reports and studies. Moreover, agencies
provide the public with basic descriptions of agency authorities,
activities and programs, along with the contact information for
the public to interact with and access that information or those
services.
Underlying
Principles
In
accordance with section 515, OMB has designed the guidelines to
help agencies ensure and maximize the quality, utility, objectivity
and integrity of the information that they disseminate (meaning
to share with, or give access to, the public). It is crucial that
information Federal agencies disseminate meets these guidelines.
In this respect, the fact that the Internet enables agencies to
communicate information quickly and easily to a wide audience not
only offers great benefits to society, but also increases the potential
harm that can result from the dissemination of information that
does not meet basic information quality guidelines. Recognizing
the wide variety of information Federal agencies disseminate and
the wide variety of dissemination practices that agencies have,
OMB developed the guidelines with several principles in mind.
First,
OMB designed the guidelines to apply to a wide variety of government
information dissemination activities that may range in importance
and scope. OMB also designed the guidelines to be generic enough
to fit all media, be they printed, electronic, or in other form.
OMB sought to avoid the problems that would be inherent in developing
detailed, prescriptive, "one-size-fits-all" government-wide guidelines
that would artificially require different types of dissemination
activities to be treated in the same manner. Through this flexibility,
each agency will be able to incorporate the requirements of these
OMB guidelines into the agency's own information resource management
and administrative practices.
Second,
OMB designed the guidelines so that agencies will meet basic information
quality standards. Given the administrative mechanisms required
by section 515 as well as the standards set forth in the PRA, it
is clear that agencies should not disseminate substantive information
that does not meet a basic level of quality. We recognize that some
government information may need to meet higher or more specific
information quality standards than those that would apply to other
types of government information. The more important the information,
the higher the quality standards to which it should be held, for
example, in those situations involving "influential scientific or
statistical information" (a phrased defined in these guidelines).
The guidelines recognize, however, that information quality comes
at a cost. Accordingly, the agencies should weigh the costs (for
example, including costs attributable to agency processing effort,
respondent burden, maintenance of needed privacy, and assurances
of suitable confidentiality) and the benefits of higher information
quality in the development of information, and the level of quality
to which the information disseminated will be held.
More
specifically, the OMB guidelines state that "agencies shall have
a basic standard of quality (including objectivity, utility, and
integrity) as a performance goal ...". We note, in the scientific
context, that in 1996 the Congress, for health decisions under the
Safe Drinking Water Act, has already adopted a basic standard of
quality for the use of science in agency decisionmaking. Under 42
U.S.C. 300g-1(b)(3)(A), an agency is directed, "to the degree that
an Agency action is based on science," to use "(i) the best available,
peer-reviewed science and supporting studies conducted in accordance
with sound and objective scientific practices; and (ii) data collected
by accepted methods or best available methods (if the reliability
of the method and the nature of the decision justifies use of the
data)." We also note that the OMB guidelines call for an additional
level of quality "in those situations involving influential scientific
or statistical information." The additional level of quality concerns
a standard of care for scientific or statistical analytical results,
a "capable of being substantially reproduced" standard that is discussed
below.
We
further note that in the 1996 amendments to the Safe Drinking Water
Act the Congress adopted a basic quality standard for the dissemination
of public information about risks of adverse health effects. Under
42 U.S.C. 300g-1(b)(3)(B), the agency is directed, "to ensure that
the presentation of information [risk] effects is comprehensive,
informative, and understandable." The agency is further directed,
"in a document made available to the public in support of a regulation
[to] specify, to the extent practicable -- (i) each population addressed
by any estimate [of applicable risk effects]; (ii) the expected
risk or central estimate of risk for the specific populations [affected];
(iii) each appropriate upper-bound or lower-bound estimate of risk;
(iv) each significant uncertainty identified in the process of the
assessment of [risk] effects and the studies that would assist in
resolving the uncertainty; and (v) peer-reviewed studies known to
the [agency] that support, are directly relevant to, or fail to
support any estimate of [risk] effects and the methodology used
to reconcile inconsistencies in the scientific data." We urge each
agency in developing its guidelines to evaluate whether adopting
or adapting these basic Congressional standards would be appropriate
for judging the quality of disseminated scientific or statistical
information.
Third,
OMB designed the proposed guidelines so that agencies can apply
them in a common-sense and workable manner. It is important that
these guidelines do not impose unnecessary administrative burdens
that would inhibit agencies from continuing to take advantage of
the Internet and other technologies to disseminate information that
can be of great benefit and value to the public. In this regard,
OMB encourages agencies to incorporate the standards and procedures
required by these guidelines into their existing information resources
management and administrative practices rather than create new and
potentially duplicative or contradictory processes. The primary
example of this is that the guidelines recognize that, in accordance
with OMB Circular A-130, agencies already have in place well-established
information quality standards and administrative mechanisms that
allow persons to seek and obtain correction of information that
is maintained and disseminated by the agency. Under the OMB guidelines,
agencies need only ensure that their own guidelines are consistent
with these OMB guidelines, and then ensure that their administrative
mechanisms satisfy the standards and procedural requirements in
the new agency guidelines. Similarly, agencies may rely on their
implementation of the Federal Government's computer security laws
(formerly, the Computer Security Act, and now the computer security
provisions of the PRA) to establish appropriate security safeguards
for ensuring the "integrity" of the information that the agencies
disseminate.
Summary
of OMB Guidelines
These
guidelines apply to Federal agencies subject to the Paperwork Reduction
Act (44 U.S.C. chapter 35). Agencies are directed to develop information
resources management procedures for reviewing and substantiating
(by documentation or other means selected by the agency) the quality
(including the objectivity, utility, and integrity) of information
before it is disseminated. In addition, agencies are to establish
administrative mechanisms allowing affected persons to seek and
obtain, where appropriate, correction of information disseminated
by the agency that does not comply with the OMB or agency guidelines.
Consistent with the underlying principles described above, these
guidelines stress the importance of having agencies apply these
standards and develop their administrative mechanisms so they can
be implemented in a common sense and workable manner. Moreover,
agencies must apply these standards flexibly, and in a manner appropriate
to the nature and timeliness of the information to be disseminated,
and incorporate them into existing agency information resources
management and administrative practices.
Section
515 denotes four substantive terms regarding information disseminated
by Federal agencies: quality, utility, objectivity, and integrity.
It is not always clear how each substantive term relates -- or how
the four terms in aggregate relate -- to the widely divergent types
of information that agencies disseminate. The guidelines provide
definitions that attempt to establish a clear meaning so that both
the agency and the public can readily judge whether a particular
type of information to be disseminated does or does not meet these
attributes.
In
the guidelines, OMB defines "quality" as the encompassing term,
of which "utility," "objectivity," and "integrity" are the constituents.
"Utility" refers to the usefulness of the information to the intended
users. "Objectivity" focuses on whether the disseminated information
is being presented in an accurate, clear, complete, and unbiased
manner, and as a matter of substance, is accurate, reliable, and
unbiased. "Integrity" refers to security -- the protection of information
from unauthorized access or revision, to ensure that the information
is not compromised through corruption or falsification. OMB modeled
the definitions of "information," "government information," "information
dissemination product," and "dissemination" on the longstanding
definitions of those terms in OMB Circular A-130, but tailored them
to fit into the context of these guidelines.
In
addition, agencies have two reporting requirements. The first report,
implemented no later than one year after the issuance of these OMB
guidelines (no later than October 1, 2002), must provide the
agency'S information quality guidelines that describe administrative
mechanisms allowing affected persons to seek and obtain, where appropriate,
correction of disseminated information that does not comply with
the OMB and agency guidelines. The second report is an annual fiscal
year report to OMB (to be first submitted on January 1, 2004) providing
information (both quantitative and qualitative, where appropriate)
on the number, nature, and resolution of complaints received by
the agency regarding its perceived or confirmed failure to comply
with these OMB and agency guidelines.
Public
Comments and OMB Response
Section
515(a) required OMB to provide the public and the Federal agencies
the opportunity to comment on these guidelines. OMB worked with
Federal agencies, through a working group and through an inter-agency
comment process, in the development of the proposed guidelines.
The proposed guidelines were published in the Federal Register
on June 28, 2001 (66 Fed. Reg. 34489) providing a public comment
period of 45 days. OMB received a total of 100 comments from academic
institutions (36), Federal agencies (26), individual members of
the public (7), associations affiliated with academia (5), associations
affiliated with medical, social science or science interests (15),
associations affiliated with Federal Government interests (4), and
associations affiliated with industry interests (7).
General
Concerns. Many comments expressed support for the idea of
government-wide quality standards for information disseminated by
Federal agencies. Comments also expressed support for OMB's commitment
to creating flexible general guidelines and to minimizing the administrative
costs and burdens that these guidelines will impose. The majority
of comments focused on two aspects of the proposed guidelines: suggestions
for placing limitations on the administrative correction mechanisms
requirements of the statute; and the need to clarify specific definitions
and other terms found in the guidelines.
Many
comments raised questions and concerns about how these guidelines
interact with existing statutes and policies, including the Paperwork
Reduction Act and the Government Performance and Results Act. We
have attempted to draft these guidelines in a way that addresses
the requirements of section 515, but does not impose a completely
new and untried set of standards upon Federal agencies. We encourage
agencies to consider the effect of relevant existing statutes and
policies in the development of their own guidelines.
Administrative
Mechanisms. These guidelines require agencies to establish
administrative mechanisms allowing affected persons to seek and
obtain, where appropriate, correction of information maintained
and disseminated by the agency that does not comply with the OMB
guidelines. Many comments suggested that limits be imposed on the
types of information that should be subject to these guidelines,
in particular, information that is disseminated by agency libraries.
OMB agrees that archival information disseminated by Federal agency
libraries (for example, Internet distribution of published articles)
should not be covered by these guidelines, given that libraries
do not endorse the information that they disseminate. Moreover,
an agency's dissemination of public filings (for example, corporate
filings with the Securities and Exchange Commission) is not covered
by these guidelines. In each of these situations, the agencies have
not authored these documents and have not adopted them as representing
the agencies' views. By disseminating these materials, the agencies
are simply ensuring that the public can have quicker and easier
access to materials that are publicly available. In developing its
implementing guidelines, and in accordance with the criteria set
forth in these guidelines, each agency should evaluate and identify
the types of information that it disseminates that will be subject
to its guidelines.
In
addition, comments also raised the concern that the guidelines would
apply to "preliminary" information, and they recommended that the
guidelines exclude such information. OMB appreciates the concerns
that these comments have raised. However, OMB does not believe that
an exclusion for "preliminary" information is necessary or appropriate.
It is still important that the quality of preliminary information
be ensured and that preliminary information be subject to the administrative
complaint-and-correction process.
A few
comments stated that affected information should be limited to information
used in agency rulemaking. While this has been the position of previous
policies which these guidelines are not intended to modify or replace
(see, e.g., section ___.36(d) in OMB Circular A-110), we believe
the plain meaning and intent of section 515 covers the larger government
information universe.
Based
on the public comments received, these guidelines allow agencies
to determine the appropriate level of correction for a complaint
received. Several comments suggested that agencies use disclaimers
to distinguish the status of information, a practice that agencies
should consider adopting as they consider their information holdings.
OMB
received detailed discussion on the requirement that agencies develop
administrative mechanisms allowing for affected persons to "seek
and obtain correction of information that does not comply with OMB's
guidelines." Members of the scientific community expressed strong
concerns about the possibility of a Federal agency that would "correct"
scientific information without carrying out the scientific analysis
to support the correction. Comments from all fields suggested in
various ways that challenging individuals should be "required to
openly state his/her relationship with the data/information (familiarity/expertise)
and provide information [as] to his/her interest in it."
Comments
also pointed out great potential for abuse of this process. As one
association summarized, "This could be seen to provide grounds for
interested parties to demand access to underlying data, to compel
the government to replicate research findings (at great expense
and with unnecessary delay), or in other ways impede, discredit,
harass or stymie research." For example, the National Oceanic and
Atmospheric Administration (NOAA) explained that they receive numerous
complaints from the public when they miss a weather forecast. "Does
this mean that the NWS [National Weather Service] could be requested
to change a forecast after the fact? Or could someone with an economic
interest challenge official observational data which could affect
the value of an insurance payment?" asks NOAA.
Overall,
OMB does not envision administrative mechanisms that would burden
agencies with frivolous claims. Instead, the correction process
should serve to address the genuine and valid needs of the agency
and its constituents without disrupting agency processes. Agencies,
in making their determination of whether or not to correct information,
may reject claims made in bad faith or without justification, and
are required to undertake only the degree of correction that they
conclude is appropriate for the nature and timeliness of the information
involved, and explain such practices in their annual fiscal year
reports to OMB.
Numerous
comments provided language to clarify or limit the term, "affected
persons." One academic institution suggested that the term, "affected
persons," reflects a criterion of "direct measurable impact with
significant personal consequence." Other academic institutions suggested
that "affected persons should not be permitted to challenge the
substance of information without showing that a qualified scientist
has found fault with its quality or integrity." Similarly, some
comments argued that the ability to correct scientific information
should be limited only to other scientists. Several associations
suggested that OMB identify the types of information that could
be challenged rather than to focus on the characteristics of a "legitimate"
challenger. OMB considered these comments at length. Our conclusion
is that "affected persons" are people who may benefit or be harmed
by the disseminated information. This includes persons who are seeking
to address information about themselves as well as persons who use
information. However, each agency should consider how persons (which
includes groups, organizations and corporations, as defined by the
Paperwork Reduction Act) will be affected by the agency's information.
Agencies should address the issue of "affected persons" in consultation
with their constituents through the public comment process that
agencies will provide after drafting their proposed guidelines and
before submitting them for OMB review.
These
guidelines require that an agency official be designated to receive
and resolve complaints regarding information that does not comply
with either the OMB guidelines or the agency's guidelines. In the
proposed guidelines, we required, with a limited exception, that
the Chief Information Officer (CIO) of the agency have this responsibility.
Of the government agencies that commented on this provision, many
pointed to their specific agency practices on information quality
and their designation of a "quality official" who was not necessarily
working under the agency CIO. Recognizing that some agencies may
have specific officials in place to address quality issues, the
final guidelines allow agencies to designate an appropriate official.
Agencies may also designate multiple officials, i.e., based on the
needs of individual agency components, as long as there is a single
official with these overall responsibilities designated at the agency
level. The authorized official also needs to consult with the CIO
on quality matters pertaining to information disseminated by the
agency.
Agencies
need to respond to complaints in a manner appropriate to the nature
and extent of the complaint. Examples of appropriate responses,
as suggested by comments, include personal contacts via letter or
telephone, form letters, press releases or mass mailings that correct
a widely disseminated error or address a frequently raised complaint.
Agencies may want to utilize other methods of response under existing
agency practices. For example, for agencies with a high volume of
complaints, it is acceptable for the agency to describe a sample
of those complaints in the annual fiscal year report to OMB. For
categories of inconsequential or trivial complaints identified in
the agency guidelines, an agency may decide that no response is
necessary. Agencies should describe to OMB as part of the annual
fiscal year report the chosen response mechanisms and how they are
working.
Definitions
and Other Terms. Section 515 denotes four substantive terms
regarding information disseminated by Federal agencies: quality,
utility, objectivity, and integrity. We have defined "quality" as
an encompassing term. The organizations and individuals that submitted
comments did not object to having "quality" defined as an encompassing
term, but suggested that we should discuss each term separately.
The principles laid out in the proposed guidelines, stated one comment,
create "subjective definitions" of the four terms. This comment
warned OMB that "subjective definitions of quality, objectivity,
utility, and integrity could cause agencies to delay data release
or disregard data for fear of challenge." Other comments expressed
similar views, or as one association observed, "Science does not
recognize a sliding scale of quality."
These
guidelines reflect OMB's determination that "quality," "utility,"
"objectivity," and "integrity" are closely interrelated concepts
in the context of these guidelines. Collectively, these terms address
the following three aspects of the information that is to be disseminated:
whether the information is useful to the intended users of the information;
whether the disseminated information is being presented in an accurate,
clear, complete, and unbiased manner in both presentation and as
a characteristic that should be inherent to quality information;
and whether the information has been protected from unauthorized
access or revision.
Some
comments stated that OMB was "exceeding the statutory mandate" and
going beyond "Congressional intent" in specifying scientific and
statistical information in these guidelines. Others felt that we
should simply acknowledge that the scientific and statistical communities
already have practices and standards for their information, rather
than create another set of standards for these information types.
OMB does not agree with those comments that said the proposed guidelines
went beyond the statute in covering statistical and scientific information.
Section 515 expressly states that its scope includes statistical
information. Moreover, section 515 has no exclusion for scientific
information, and in many respects it is very similar to (and overlaps
with) statistical information. OMB, however, does appreciate the
concerns that the comments raised about the guidelines not creating
another set of standards for statistical and scientific information.
Our guidelines do not seek to impose new standards on these communities,
but to reiterate the standards that are already held in those communities.
Recognizing
public interest in medical and public health information, we have
specifically added a provision stating, "Agencies shall adopt specific
standards of quality that are appropriate for the various categories
of information they disseminate." For example, OMB encourages agencies,
in crafting their agency-specific guidelines, to promote objectivity
in information quality in ways that protect the confidentiality
of research subjects and encourage public participation in research.
These concerns are particularly salient in medical and public health
research.
A number
of comments regarded our discussion of ensuring that scientific
information be "substantially reproducible" as requiring agencies
to replicate original data and to perform independent analysis upon
all scientific information disseminated by the agency. We have responded
to these concerns in a number of ways. First, we make it clear that
what we now refer to as the "capable of being substantially reproduced"
standard applies to analytical results that are disseminated, and
does not apply to the original or supporting data. Thus, replication
of original data is not required. Second, the "capable of being
substantially reproduced" standard is applicable only to "influential"
scientific and statistical information as defined in the guidelines.
Third, the guidelines call for the agency to determine that "influential"
analytical results be capable of being substantially reproducible
by independent analysis. We intend this standard to say that, if
appropriately qualified persons used the same or a similar methodology,
they would be expected to achieve similar findings and results.
Based
on the concerns expressed in the comments, we expanded upon our
discussion of "capable of being substantially reproduced" in our
definition of "objective," and added two explanatory definitions.
We state, in paragraphs V.3.B, V.9, and V.10:
In
addition, "Objectivity" involves a focus on ensuring accurate, reliable,
and unbiased information. In a scientific or statistical context,
the original or supporting data shall be generated, and the analytical
results shall be developed, using sound statistical and research
methods.
- If
the results have been subject to formal, independent, external
peer review, the information can generally be considered of acceptable
objectivity.
- In
those situations involving influential scientific or statistical
information, the results must be capable of being substantially
reproduced, if the original or supporting data are independently
analyzed using the same models. Reproducibility does not mean
that the original or supporting data have to be capable of being
replicated through new experiments, samples or tests.
- Making
the data and models publicly available will assist in determining
whether analytical results are capable of being substantially
reproduced. However, these guidelines do not alter the otherwise
applicable standards and procedures for determining when and how
information is disclosed. Thus, the objectivity standard does
not override other compelling interests, such as privacy, trade
secret, and other confidentiality protections.
"Influential"
when used in the phrase "influential scientific or statistical information"
means the agency expects that information in the form of analytical
results will likely have an important effect on the development
of domestic or international government or private sector policies
or will likely have important consequences for specific technologies,
substances, products or firms.
"Capable
of being substantially reproduced" means that independent reanalysis
of the original or supporting data using the same methods would
generate similar analytical results, subject to an acceptable degree
of imprecision.
As
a general matter, in the scientific and research context, we regard
technical information that has been subjected to formal, independent,
external peer review as presumptively objective. An example of a
formal independent external peer review is the review process used
by scientific journals. However, depending on the nature and timeliness
of the information involved, an agency may decide that peer review
is not necessary or appropriate. On the other hand, in those situations
involving influential scientific or statistical information, the
substantial reproducibility standard is added as a quality standard
above and beyond some peer review quality standards. In the definition
of "influential," when used in the phrase "influential scientific
or statistical information," we note that the manner in which people
perceive the scientific or statistical information can have important
consequences for specific policies, technologies, substances, products,
and firms.
Based
on concerns with the "substantially reproducible" standard, a number
of comments suggested that OMB should repropose this standard for
additional public comment, rather than going final at this time.
While, in deference to the statutory deadline, OMB is issuing the
"capable of being substantially reproduced" standard (paragraphs
V.3.B, V.9, and V.10), OMB is doing so on an interim final basis.
We specifically request public comments on this standard by October
28, 2001. In addition, OMB wants to stress that the guidelines published
today should be understood as a beginning of an evolutionary process
that will include draft agency guidelines, public comment, final
agency guidelines, development of experience with OMB and agency
guidelines, and continued refinement of both OMB and agency guidelines.
OMB
modeled the draft definitions of "information," "government information,"
"information dissemination product," and "dissemination" on the
longstanding definitions of those terms in OMB Circular A-130, but
tailored them to fit into the context of these guidelines. Information
that is disseminated on behalf of an agency (through a contract
or a grant) is considered to be sponsored by the agency and is subject
to these guidelines. Consistent with the PRA concept of agency "sponsorship"
of a collection of information, information is considered to be
disseminated on behalf of an agency by a contractor or grantee if
the dissemination is done at the agency's specific request or with
the agency's specific approval. See 5 C.F.R. 1320.3(d). Finally,
it should be noted that these guidelines focus primarily on the
dissemination of substantive information (i.e. reports, studies,
summaries) rather than information pertaining to basic agency operations.
We
have clarified two terms for the final guidelines. The proposed
guidelines included "opinions" in the definition of "information."
We agree with comments that indicated agencies should not be accountable
for correcting someone's opinion, but in the agency's presentation
of the information, it should be clear that what is being offered
is someone's opinion rather than facts or the agency's views. "Opinion"
has therefore been removed from the definition of "information"
in the final guidelines. The definition for "dissemination" was
also revised after discussions with two Federal agencies that correspond
frequently with individual members of the public regarding their
participation in the agency's programs. In addition, in the definition
of "dissemination," we changed the exclusion for "judicial process"
to "adjudicative process" to make it clear that these guidelines
do not apply to the issuance of agency adjudicative decisions.
Reporting
Requirements. Agencies have two reporting requirements.
The first report, taking effect no later than one year after the
issuance of these OMB guidelines, must provide the agency's information
quality guidelines that describe administrative mechanisms allowing
affected persons to seek and obtain, where appropriate, correction
of disseminated information that does not comply with these OMB
guidelines. During the year that agencies have to complete their
agency guidelines, agencies must publish the draft reports in the
Federal Register for a period of public comment, and no later
than nine months after the issuance of OMB's guidelines, submit
their draft reports to OMB for review. Upon completion of OMB's
review, final agency guidelines must be published in the Federal
Register and made available through the agency website. The
entire process must be completed by no later than one year after
the issuance of the OMB guidance (no later than October 1,
2002).
The
second report is an annual fiscal year report to OMB (to be first
submitted on January 1, 2004) providing information on the number,
nature, and resolution of complaints received by the agency regarding
its perceived or confirmed failure to comply with these OMB and
agency guidelines. Regarding the proposed guidelines, we received
detailed comments on the required report to OMB describing the number
and nature of complaints received by the agency and how such complaints
were resolved. Two Federal agencies stated that it would be burdensome
to report to OMB on every single complaint they received and responded
to, particularly because many of the complaints may be received
in phone calls and given informal responses that address the callers'
concerns. Recognizing that agencies may deal with large volumes
of complaints on particular types of information disseminated by
the agency, OMB's guidelines allow the agency to provide qualitative
and/or quantitative descriptions of complaints received and how
they were resolved (or not). OMB also recognizes that a large number
of comments about a specific document may only demonstrate that
the information is controversial, not that its quality is flawed.
In
conclusion, issuance of these final guidelines meets the statutory
requirement that section 515 imposed on OMB. As we stated earlier
in this preamble, and in connection with the proposed guidelines,
OMB has sought in developing these guidelines to make them flexible
enough so that Federal agencies can apply them in a common sense,
workable, and appropriately tailored manner to the wide variety
of dissemination activities that the Federal Government undertakes.
In addition, in drafting guidelines that will apply on a government-wide
basis, OMB has been sensitive to the problem of unintended consequences
and has tried to anticipate and address issues that could arise
during the implementation of these guidelines. In this respect,
the public and agency comments that we received on the proposed
guidelines were very helpful and are greatly appreciated. As we
explained above, we made a number of revisions to the guidelines
to address the concerns raised in the comments, and we also believe
that these and other concerns can be addressed as well in the implementing
guidelines that each agency will develop in the coming months. In
addition, OMB is issuing the "capable of being substantially reproduced"
standard (paragraphs V.3.B, V.9, and V.10) on an interim final basis.
We specifically request public comments on this standard over the
next 30 days. Moreover, over time as the agencies and the public
gain further experience with the OMB guidelines, we would appreciate
receiving any suggestions for how OMB could improve them. Just as
OMB requested public comment before issuing these final guidelines,
OMB will refine these guidelines as experience develops and further
public comment is obtained.
Dated:
September 24, 2001
/s/
John D. Graham
Administrator, Office of Information and Regulatory Affairs
Guidelines
for Ensuring and Maximizing the Quality, Objectivity, Utility, and
Integrity of Information Disseminated by Federal Agencies.
I.
OMB Responsibilities. Section 515 of the Treasury and General
Government Appropriations Act for FY2001 (Public Law 106-554) directs
the Office of Management and Budget to issue government-wide guidelines
that provide policy and procedural guidance to Federal agencies
for ensuring and maximizing the quality, objectivity, utility, and
integrity of information, including statistical information, disseminated
by Federal agencies.
II.
Agency Responsibilities. Section 515 directs agencies subject
to the Paperwork Reduction Act (44 U.S.C. 3502(a)) to --
- Issue
their own information quality guidelines ensuring and maximizing
the quality, objectivity, utility, and integrity of information,
including statistical information, disseminated by the agency
no later than one year after the date of issuance of the OMB guidelines;
- Establish
administrative mechanisms allowing affected persons to seek and
obtain correction of information maintained and disseminated by
the agency that does not comply with these OMB guidelines; and
- Report
to the Director of OMB the number and nature of complaints received
by the agency regarding agency compliance with these OMB guidelines
concerning the quality, objectivity, utility, and integrity of
information and how such complaints were resolved.
III.
Guidelines for Ensuring and Maximizing the Quality, Objectivity,
Utility, and Integrity of Information Disseminated by Federal Agencies.
- Overall,
agencies shall adopt a basic standard of quality (including objectivity,
utility, and integrity) as a performance goal and should take
appropriate steps to incorporate information quality criteria
into agency information dissemination practices. Quality is to
be ensured and established at levels appropriate to the nature
and timeliness of the information to be disseminated. Agencies
shall adopt specific standards of quality that are appropriate
for the various categories of information they disseminate.
- As
a matter of good and effective agency information resources management,
agencies shall develop a process for reviewing the quality (including
the objectivity, utility, and integrity) of information before
it is disseminated. Agencies shall treat information quality as
integral to every step of an agency's development of information,
including creation, collection, maintenance, and dissemination.
This process shall enable the agency to substantiate the quality
of the information it has disseminated through documentation or
other means appropriate to the information.
- To
facilitate citizen review, agencies shall establish administrative
mechanisms allowing affected persons to seek and obtain, where
appropriate, timely correction of information maintained and disseminated
by the agency that does not comply with OMB or agency guidelines.
These administrative mechanisms shall be flexible, appropriate
to the nature and timeliness of the disseminated information,
and incorporated into agency information resources management
and administrative practices.
- The
agency's pre-dissemination review, under paragraph III.2, shall
apply to information that the agency first disseminates on or
after October 1, 2002. The agency's administrative mechanisms,
under paragraph III.3, shall apply to information that the agency
disseminates on or after October 1, 2002, regardless of when
the agency first disseminated the information.
IV.
Agency Reporting Requirements.
- Agencies
must designate the Chief Information Officer or another official
to be responsible for agency compliance with these guidelines.
- The
agency shall respond to complaints in a manner appropriate to
the nature and extent of the complaint. Examples of appropriate
responses include personal contacts via letter or telephone, form
letters, press releases or mass mailings that correct a widely
disseminated error or address a frequently raised complaint.
- Each
agency must prepare a draft report, no later than April 1,
2002, providing the agency's information quality guidelines and
explaining how such guidelines will ensure and maximize the quality,
objectivity, utility, and integrity of information, including
statistical information, disseminated by the agency. This report
must also detail the administrative mechanisms developed by that
agency to allow affected persons to seek and obtain appropriate
correction of information maintained and disseminated by the agency
that does not comply with the OMB or the agency guidelines.
- The
agency must publish a notice of availability of this draft report
in the Federal Register, and post this report on the agency's
website, to provide an opportunity for public comment.
- Upon
consideration of public comment and after appropriate revision,
the agency must submit this draft report to OMB for review regarding
consistency with these OMB guidelines no later than July 1, 2002.
Upon completion of that OMB review and completion of this report,
agencies must publish notice of the availability of this report
in the Federal Register, and post this report on the agency's
web site no later than October 1, 2002.
- On
an annual fiscal-year basis, each agency must submit a report
to the Director of OMB providing information (both quantitative
and qualitative, where appropriate) on the number and nature of
complaints received by the agency regarding agency compliance
with these OMB guidelines and how such complaints were resolved.
Agencies must submit these reports no later than January 1 of
each following year, with the first report due January 1, 2004.
V.
Definitions.
- "Quality"
is an encompassing term comprising utility, objectivity, and integrity.
Therefore, the guidelines sometimes refer to these four statutory
terms, collectively, as "quality."
- "Utility"
refers to the usefulness of the information to its intended users,
including the public. In assessing the usefulness of information
that the agency disseminates to the public, the agency needs to
consider the uses of the information not only from the perspective
of the agency but also from the perspective of the public. As
a result, when reproducibility and transparency of information
are relevant for assessing the information's usefulness from the
public's perspective, the agency must take care to ensure that
reproducibility and transparency have been addressed in its review
of the information.
- "Objectivity"
involves two distinct elements, presentation and substance.
- "Objectivity"
includes whether disseminated information is being presented
in an accurate, clear, complete, and unbiased manner. This involves
whether the information is presented within a proper context.
Sometimes, in disseminating certain types of information to
the public, other information must also be disseminated in order
to ensure an accurate, clear, complete, and unbiased presentation.
Also, the agency needs to identify the sources of the disseminated
information (to the extent possible, consistent with confidentiality
protections) and, in a scientific or statistical context, the
supporting data and models, so that the public can assess for
itself whether there may be some reason to question the objectivity
of the sources. Where appropriate, supporting data should have
full, accurate, transparent documentation, and error sources
affecting data quality should be identified and disclosed to
users.
- In
addition, "objectivity" involves a focus on ensuring accurate,
reliable, and unbiased information. In a scientific or statistical
context, the original or supporting data shall be generated,
and the analytical results shall be developed, using sound statistical
and research methods.
- If
the results have been subject to formal, independent, external
peer review, the information can generally be considered of
acceptable objectivity.
- In
those situations involving influential scientific or statistical
information, the results must be capable of being substantially
reproduced, if the original or supporting data are independently
analyzed using the same models. Reproducibility does not mean
that the original or supporting data have to be capable of
being replicated through new experiments, samples or tests.
- Making
the data and models publicly available will assist in determining
whether analytical results are capable of being substantially
reproduced. However, these guidelines do not alter the otherwise
applicable standards and procedures for determining when and
how information is disclosed. Thus, the objectivity standard
does not override other compelling interests, such as privacy,
trade secret, and other confidentiality protections.
- "Integrity"
refers to the security of information -- protection of the information
from unauthorized access or revision, to ensure that the information
is not compromised through corruption or falsification.
- "Information"
means any communication or representation of knowledge such as
facts or data, in any medium or form, including textual, numerical,
graphic, cartographic, narrative, or audiovisual forms. This definition
includes information that an agency disseminates from a web page,
but does not include the provision of hyperlinks to information
that others disseminate. This definition does not include opinions,
where the agency's presentation makes it clear that what is being
offered is someone's opinion rather than fact or the agency's
views.
- "Government
information" means information created, collected, processed,
disseminated, or disposed of by or for the Federal Government.
- "Information
dissemination product" means any book, paper, map, machine-readable
material, audiovisual production, or other documentary material,
regardless of physical form or characteristic, an agency disseminates
to the public. This definition includes any electronic document,
CD-ROM, or web page.
- "Dissemination"
means agency initiated or sponsored distribution of information
to the public (see 5 C.F.R. 1320.3(d) (definition of "Conduct
or Sponsor"). Dissemination does not include distribution limited
to government employees or agency contractors or grantees; intra-
or inter-agency use or sharing of government information; and
responses to requests for agency records under the Freedom of
Information Act, the Privacy Act, the Federal Advisory Committee
Act or other similar law. This definition also does not include
distribution limited to correspondence with individuals or persons,
press releases, archival records, public filings, subpoenas or
adjudicative processes.
- "Influential"
when used in the phrase "influential scientific or statistical
information" means the agency expects that information in the
form of analytical results will likely have an important effect
on the development of domestic or international government or
private sector policies or will likely have important consequences
for specific technologies, substances, products or firms.
- "Capable
of being substantially reproduced" means that independent reanalysis
of the original or supporting data using the same methods would
generate similar analytical results, subject to an acceptable
degree of imprecision.
|