Recommendations
and Guidelines for the Use of Digital Image Processing in the Criminal
Justice System
Scientific
Working Group on Imaging Technologies (SWGIT)
Version 1.2, June 2002
Purpose.......Background.......Introduction.......Definitions.......Image
Enhancement
Image Restoration.......Image
Compression.......Quantitative
Image Analysis
Guidelines for Digital Image Processing SOPs.......Appendix
A.......Appendix B
Appendix C
Purpose
The purpose
of this document is to provide recommendations and guidelines for
the use of digital image processing in the criminal justice system.
The objective is to ensure the successful introduction of forensic
imagery as evidence in a court of law. This document includes brief
descriptions of advantages, disadvantages, and potential limitations
of each major process.
Background
Digital image
processing is an accepted practice in forensic science. It is the
position of the Scientific Working Group on Imaging Technologies
(SWGIT) that any changes to an image made through digital image
processing are acceptable in forensic applications provided the
following criteria are met:
- The original
image is preserved
- The processing
steps are logged when they include techniques other than those
used in a traditional photographic darkroom
- The end result
is presented as an enhanced image, which may be reproduced by
applying the logged steps to the original image
- The recommendations
of this document are followed
Introduction
This document
addresses digital image processing and related legal considerations
in the following four categories:
- Image enhancement
- Image restoration
- Image compression
- Image analysis
When using digital
image processing techniques, use caution to avoid the introduction
of unexplainable artifacts that add misleading information to the
image and the loss of image detail that could lead to an erroneous
interpretation. Any processing techniques should be applied only
to the working image.
The successful
introduction of forensic imagery as evidence in a court of law is
dependant upon the following four legal tests:
- Reliability
- Reproducibility
- Security
- Discovery
Definitions
Apple QuickTime
Movie Format (MOV file format): Apple file format for storing
and displaying compressed video sequences.
Artifact:
Any visible feature or distortion in a recorded image or output
image that is not present in the corresponding imaged object or
input image. Image artifacts can be introduced inadvertently by
hardware or software or intentionally by an operator. The latter
type includes annotation or other direct alteration of an image
in order to clarify or call attention to some particular image content.
Artifacts introduced by hardware and software generally degrade
an image and, if severe enough, can impair interpretation.
AVI (Audio/Video
Interlaced): Microsoft file format for storing and displaying compressed
video sequences.
Compression
ratio: The size of an image data file before compression divided
by the file size after compression.
Copy image:
A reproduction of information contained in a primary or original
image.
Deinterlacing:
Separating an interlaced frame into two separate fields.
Discovery:
The criminal defendant's right to confront and challenge the evidence.
Forensic:
The use or application of scientific knowledge to a point of law,
especially as it applies to the investigation of crime.
GIF (Graphical
Interchange Format): A lossless compression file format commonly
used for graphic images.
Image analysis:
The extraction of quantitative information from an image beyond
which is readily apparent through visual examination.
Image averaging:
The process of averaging together similar images, such as sequential
video frames, to reduce noise in stationary scenes.
Image compression:
The process of reducing the size of a data file.
Image enhancement:
Any process intended to improve the visual appearance of an image.
Image processing:
Any activity that transforms an input image into an output image.
Note: Image processing does not mean that the input image is overwritten
during the process. Forensic image processing should be performed
only on working images.
Image restoration:
Any process applied to an image that has been degraded by a known
cause, such as defocus or motion blur, so the effects of that degradation
are partially or totally removed.
Image synthesis:
Any process that renders an image through the use of computer graphics
techniques for illustrative purposes (e.g., age progression, facial
reconstruction, accident/crime scene reconstruction). This subject
is beyond the scope of this document.
Interlaced
scan: Video image format. The video frame consists of two fields.
The first field contains all the odd-numbered horizontal lines and
the second field all the even-numbered lines. All standard TV video
signals used in North America and elsewhere under the NTSC (RS-170,
CCIR) standard are in the interlaced format.
Interpolation:
A process by which the apparent resolution of an image is increased.
In most cases the software mathematically averages adjacent pixel
densities and places a pixel of that density between the two.
JPEG
(Joint Photographic Experts Group): A lossy image compression process.
Users can set their own quality settings on a sliding scale within
the application software.
JPEG 2000:
An image compression process currently under development.
LZW (Lempel-Ziv-Welch):
A lossless compression process used by the TIFF and GIF file formats.
MPEG
(Motion Pictures Experts Group): Similar to JPEG, a standard compression
algorithm used to compress video and audio sequences.
Original
image: An accurate and complete replica of the primary image,
regardless of media. For film and analog video, the primary image
is the original image.
Primary image:
Refers to the first instance in which an image is recorded onto
any media that is a separate, identifiable object or objects. Examples
include a digital image recorded on a flash card or a digital image
downloaded from the Internet.
Progressive
scan: (Noninterlace) Video in which each image frame contains
information from every horizontal scan line of the imaging sensor.
Reliability:
The extent to which information can be depended upon.
Reproducibility:
The extent to which a process yields the same results on repeated
trials.
Security:
The extent to which the evidence has been preserved and safeguarded.
TIFF
(Tagged Image File Format): A standardized image file exchange format.
It is widely supported by both hardware and software manufacturers
and is platform independent. Can be lossless or lossy.
Working image:
Any image subjected to processing.
Image Enhancement
Image enhancement
is any process intended to improve the visual appearance of an image.
This includes processes that have a direct counterpart in the conventional
silver-based photographic laboratory and those that can be accomplished
only by using a computer.
Traditional Enhancement
Techniques
Traditional
enhancement techniques are techniques that have direct counterparts
in traditional darkrooms. They include brightness and contrast adjustment,
color balancing, cropping, and dodging and burning. These traditional
and acceptable forensic techniques are used to achieve an accurate
recording of an event or object.
Brightness adjustment
is used when the image is too bright or too dark. If the image is
made too bright, there is a risk of loss of detail in light areas.
If the image is made too dark, there is a risk of loss of detail
in the dark areas.
Color balancing
is the adjustment of the color components of an image. The purpose
of color balancing is to render the colors in the scene faithfully.
Improper color balance adjustment can render colors inaccurately,
and objects will appear to have the wrong color when compared to
the actual subject.
Contrast adjustment
is used when the image lacks sufficient contrast. If the image contrast
is increased too much, there is a risk of loss of detail in both
light and dark areas.
Cropping is
used to remove that portion of the image that is outside the area
of interest.
Dodging and
burning have the same effect as brightness adjustment but are used
in localized areas.
Spotting traditionally
has been used to remove artifacts due to dust and scratches on the
negatives, but it is not considered to be an acceptable practice
on any forensic image.
Note: The use
of spotting and cropping techniques may come under scrutiny in a
court of law. Specific agency policies should address the use of
these techniques.
Nontraditional Enhancement
Techniques
Some nontraditional
image enhancement processes are used and accepted by a variety of
scientific fields such as medicine, aerospace, and cartography.
These processes have no direct counterpart within traditional silver-based
photography. In fact, only recently have they been applied within
the forensic environment; therefore, their general acceptance may
be subject to challenge. Examples of nontraditional processes discussed
here are color processing, linear filtering, nonlinear contrast
adjustments, pattern noise reduction, and random noise reduction.
Color processing
includes color space transformations, pseudocoloring, and hue and
saturation adjustments. These techniques can be used to modify the
color characteristics of objects within an image. Caution: Application
of these techniques can compromise the color fidelity of the image.
Linear
filtering techniques include sharpening, deblurring, edge enhancement,
and deconvolution. They are used to increase the contrast of small
detail in an image. If a low degree of enhancement is used, the
image will remain an accurate representation of the scene. If a
high degree of enhancement is used, the image may no longer be an
accurate representation of the overall scene, though still may be
useful as an adjunct for interpretation of small details. Caution:
A high degree of enhancement can also increase the visibility of
existing noise and artifacts. Examples of noise include film grain,
snow appearing on a TV screen, or random color dots. (Figure 1)
|
Figure
1
This example illustrates the effects
of linear filtering. Left: original image; Middle: blurred
image; Right: sharpened image.
Click to enlarge image.
|
Nonlinear
contrast adjustments include gamma correction, grayscale transformation,
curves, and look-up tables. They are an extension of traditional
photographic sensitometric techniques and are used to adjust the
contrast in selected brightness ranges within the image. For example,
details may be brought out in the shadow areas without affecting
the highlight areas. Caution: A severe adjustment can cause loss
of detail, color reversal, and the introduction of artifacts. (Figure
2)
|
Figure
2 This
example shows nonlinear contrast adjustments. Left: original
image; Middle: enhancement of shadow and highlight areas,
at the expense of midrange tones; Right: enhancement of midrange
tones, at the expense of shadow and highlight areas.
Click to enlarge image.
|
Pattern noise
reduction filters identify repeating patterns in the image and allow
the user to selectively remove them. This type of filter can be
used to remove patterns such as fabric weaves, window screens, security
patterns, and halftone dots. Caution: Overuse of this technique
can cause selective removal of relevant image detail.
Random noise
reduction techniques include such filters as low pass, blurring,
median, and despeckling. They are used to reduce the contrast of
small detail in the image in order to suppress random noise. Caution:
Overuse of this technique can cause loss of relevant detail.
Considerations for
the Application of Image Enhancement Techniques
Question:
What type of image must not be enhanced?
Answer: A primary or original image.
Discussion:
Because a primary or original image represents the first instance
where the image is recorded onto any media, or it is an accurate
and complete replica of the primary image, it must not be altered
or modified. Enhancements are performed only on working images.
Question:
Is it necessary to document the enhancement process used to produce
an enhanced image?
Answer: The need to document the enhancement process is determined
by the process used.
Discussion: Documentation of enhancement steps is not necessary
when using traditional darkroom techniques. When using nontraditional
image enhancement techniques such as unsharp masking or random noise
reduction, enhancement steps should be documented in the case notes
in sufficient detail to enable another comparatively trained individual
to repeat the steps and produce the same output when the image is
subjected to image analysis.
Question:
In a legal setting, are enhanced images discoverable?
Answer: Yes.
Discussion:
All images may be discoverable. In cases where images are enhanced,
both the original and the enhanced image, along with associated
documentation, may be discoverable.
Question:
Who is responsible for testifying about an enhanced image?
Answer: The person doing the enhancement or a person skilled
in and knowledgeable about the enhancement process that was used.
Discussion:
The person who performed the enhancement is best qualified to testify
about the enhancement techniques used. However, there may be occasions
where the court will require the assistance of additional subject-matter
experts.
Question:
Are there legal ramifications associated with the software used
specifically for image enhancement?
Answer: Yes.
Discussion:
Some considerations may include:
- Have the
particular functions within the software been accepted by the
scientific community?
- Does the
software perform as the manufacturer purports?
- Has the use
of this software been reviewed by the judicial system?
- Does the
software have "-ins" that are produced by another manufacturer?
- Is the enhancement
process repeatable and reliable?
Image Restoration
Image restoration
is any process applied to an image that has been degraded by a known
cause (e.g., defocus or motion blur) to partially or totally remove
the effects of that degradation.
Limitations
are imposed on this technique by any noise in the image and by the
fact that information that has been totally lost cannot be replaced.
Often partial restoration can be successful even when total restoration
is impossible.
Restoration Techniques
Blur removal
is a filtering technique designed to partially or completely remove
an image blur imposed by a known cause. It differs from the image
enhancement filtering processes because the blur removal filter
is designed specifically for the process that blurred the particular
image under examination. Examples include defocus and motion blur,
since these blurring phenomena can be described mathematically.
Thus, a specific filter can be designed to compensate for each blur.
The degree to which a blur can be successfully removed is limited
by noise in the image, the accuracy with which the actual blurring
process can be described mathematically, and the fact that information
has been totally lost and cannot be replaced. Often partial deblurring
can be successful even when total deblurring is impossible.
Color balancing
is the extension of grayscale linearization to a color image. It
is the adjustment of the color components of an image. The purpose
of color balancing is to render the colors in the scene faithfully.
For example, a color test target having known colors can be placed
in the scene prior to recording the image. Then a grayscale transformation
(nonlinear contrast stretch) can be designed for each color channel
(red, green, and blue) to place the different colors on the test
target in their proper relationship. It is commonly assumed that
the color of other objects in the scene will be rendered accurately
as well. Improper color balance can render colors inaccurately,
causing objects to appear to have the wrong color.
Geometric restoration
is the removal of geometric distortion from an image. Its purpose
is to restore the proper spatial relationships among the objects
in the scene. It can be used for the removal of geometric distortion,
such as that introduced by a curved mirror or a fish-eye lens. It
differs from image warping in that the geometric transformation
is designed specifically for the process that distorted the particular
image under examination. The degree to which geometric distortion
can be successfully restored is limited by the accuracy with which
the actual distortion process can be described mathematically and
the fact that information that has been totally lost (e.g., hidden
behind another object or obscured from the camera) cannot be replaced.
Often partial geometric restoration can be successful even when
exact geometric restoration is impossible.
Grayscale linearization
is the adjustment of brightness relationships among the objects
in a scene. The purpose of grayscale linearization is to render
faithfully the different brightness values in the scene. For example,
a monochrome test target having known gray values can be placed
in the scene prior to recording the image. Then a grayscale transformation
(nonlinear contrast stretch) can be designed to place the different
gray values on the test target in their proper relationship. It
is commonly assumed that the other objects in the scene will be
put in their proper brightness relationship as well. Improper grayscale
linearization can render brightness values inaccurately so that
objects may appear brighter or darker than they actually appeared
when the image was recorded.
Warping, unlike
other image restoration processes, changes the spatial relationships
among the objects in an image. It is analogous to printing a photograph
on a rubber sheet, then stretching the sheet in different directions
and then tacking it down. Warping can be used, for example, to remove
perspective from an image or to "unroll" a poster that
was wrapped around a pole. Used improperly, it can distort the natural
appearance of the objects in a scene.
Considerations
for the Application of Image Restoration Techniques
Question:
What type of image must not be restored?
Answer: A primary or original image.
Discussion: Because a primary or original image represents
the first instance where the image is recorded onto any media, or
it is an accurate and complete replica of the primary image, it
must not be altered or modified.
Question:
Is it necessary to document the restoration process?
Answer: Yes.
Discussion: Documentation of restoration steps is always
required.
Question:
Are restored images discoverable in legal proceedings?
Answer: Yes.
Discussion: All images may be discoverable. In cases where
images are restored, both the original and the restored image, along
with associated documentation, may be discoverable.
Question:
Who is responsible for testifying about a restored image?
Answer: The person doing the restoration or a person skilled
in and knowledgeable about the restoration process that was used.
Discussion: The person who performed the restoration is best
qualified to testify about the restoration techniques used. However,
there may be occasions when the court will require the assistance
of additional subject-matter experts.
Question:
Are there legal ramifications associated with the software used
specifically for image restoration?
Answer: Yes.
Discussion: Some considerations may include:
- Have the
particular functions within the software been accepted by the
scientific community?
- Does the
software perform as the manufacturer purports?
- Has the use
of this software been reviewed by the judicial system?
- Does the
software have "plug-ins" that are produced by another
manufacturer?
- Is the restoration
process repeatable and reliable?
- Has the degradation
process been accurately modeled?
Image Compression
Digital images
produce a large amount of data to be stored. Image compression techniques
reduce the storage requirements by making image data files smaller.
Compression Processes
Lossless compression
reduces file size by removing redundant information. Because the
redundant information can be replaced in order to display the image,
lossless compression results in no loss of information. Lossless
compression does not alter the content of an image when it is decompressed.
An example of a file format that uses lossless compression is the
graphical interchange format (GIF).
Lossy compression
achieves greater reduction in file size by removing both redundant
and irrelevant information. Because the irrelevant information (as
determined by the compression algorithm) cannot be replaced upon
reconstruction of an image for display, lossy compression results
in some loss of image content as well as the introduction of artifacts.
The degradation occurs each time the image is saved in a lossy file
format. Higher compression ratios result in the loss of more information.
Normally the degree of compression can be specified. Depending upon
the application, lossy compression may render an image less useful.
Caution: Lossy compression should be used with care to avoid material
degradation of the image. Additionally, the compression settings
used by one camera or software program may not be the same as the
compression settings used by another camera or software program.
The
commonly used joint photographic experts group (JPEG) image storage
format employs lossy image compression. It is applied to the image
in 8-pixel by 8-pixel blocks. Normally, the degree of compression
can be specified prior to storing the image. At high compression
ratios, JPEG could remove important image detail and introduce blocking
artifacts as the block boundaries become visible. (Figure 3) Digital
cameras often create digital images in JPEG format, so that some
lossy compression is unavoidable. The degree of compression should
be set low enough that important image content is not lost or obscured
by artifacts.
|
Figure
3 Left:
original image; Middle: the result of JPEG compression (compression
ratio = 15:1); Right: the result of edge enhancement after
compression.
Click to enlarge image.
|
Considerations
for the Application of Image Compression Techniques
Question:
What type of image must not be compressed?
Answer: It depends on the end use and need.
Discussion:
In instances where the primary or original image is already compressed,
it should not be further compressed using lossy compression processes;
additional data will be lost. Sources of compressed primary images
may include electronic booking photographs, some types of digital
camera images, and images downloaded from the Internet or E-mail.
The file format is not an indicator of the compression history for
an image. For example, a TIFF file may have been previously compressed
in a lossy file format (JPEG). Be aware that the end use of any
image may change over time, and the use of compression may become
problematic. Images intended for laboratory analysis should not
be compressed using a lossy process.
Question:
Is it necessary to document the compression history of an image?
Answer: It depends on the intended use of the image.
Discussion: The type and degree of compression may become
an issue in a judicial proceeding. Documentation may be necessary
in a court of law when argued that compression might have introduced
artifacts or relevant information was lost.
Question:
Who is responsible for testifying about a compressed image?
Answer: The person doing the compression can testify about
the settings used to compress an image.
Discussion: Questions concerning the actual compression process
should be referred to individuals who possess sufficient technical
expertise to explain the specific process.
Quantitative
Image Analysis
Quantitative
image analysis is the process used to extract quantitative (measurable)
data from an image, whereas cognitive image analysis is the process
used to extract visual information from an image. This section discusses
quantitative analysis only.
Quantitative
image analysis requires proper calibration of the image. In a digital
image the pixel spacing must be known in order to extract accurate
size measurements. Objects that are different distances from the
camera will have different pixel spacing. The accuracy of the extracted
measurements will depend upon the accuracy of calibration. Caution:
The use of image compression can degrade the accuracy of subsequent
quantitative image analysis.
An example of
a quantitative image analysis might be if a circular object in an
image includes 314 pixels, and the area covered by a single pixel
is one square millimeter, then one can conclude that the area of
the object is 314 square millimeters. Similarly, if the distance
between the adjacent pixels in an image of a document is 0.02 inches,
and the length of the document is 340 pixels, then it must be 340
times 0.02, or 6.8 inches long. These examples do not consider perspective
distortion.
Quantitative Image Analysis Techniques
Colorimetry
is the quantification of the color of an object.
Image authentication
verifies that the original image has not been altered.
Photogrammetry
is the science involving methods, techniques, and analytical procedures
used to make accurate measurements of distances and/or sizes of
objects from photographic images.
Photometry is
the measurement of light values of objects in an image.
Considerations for the
Application of Image Analysis Techniques
Question:
Which types of image should be subjected to quantitative image analysis?
Answer: A working image.
Discussion: Because a primary or original image represents
the first instance where the image is recorded onto any media, or
it is an accurate and complete replica of the primary image, it
must not be altered or modified.
Question:
Is it necessary to document quantitative image analysis?
Answer: Yes.
Discussion: Documentation of quantitative image analysis
steps is required in sufficient detail to enable another comparably
trained individual to repeat the steps and produce the same conclusions.
Question:
Are analyzed images discoverable?
Answer: Yes.
Discussion:
All analyzed images, documentation, and conclusions may be discoverable.
Question:
Who is responsible for testifying about an analyzed image?
Answer: The person doing the analysis or a person skilled
in and knowledgeable about the analysis performed.
Discussion: The person who performed the analysis is best
qualified to testify concerning the techniques used. However, there
may be occasions where the court will require the assistance of
additional subject-matter experts.
Question:
Are there legal ramifications associated with the software used
specifically for image analysis?
Answer: Yes.
Discussion: Some considerations may include:
- Have the
particular functions within the software been accepted by the
scientific community?
- Does the
software perform as the manufacturer purports?
- Has the use
of this software been reviewed by the judicial system?
- Does the
software have "plug-ins" that are produced by another
manufacturer?
- Is the analysis
repeatable and reliable?
Additional
Imaging Considerations
Question:
Where does image processing take place, in the field or in a controlled
environment?
Answer: Both.
Discussion:
Whereas most image processing takes place in a controlled environment,
some image processing, such as photogrammetry and image compression,
may take place in the field.
Question:
Who performs image processing?
Answer: Photographers, analysts, and technicians.
Discussion:
The person performing the processing must be properly trained.
Question:
What are file management processes?
Answer: File management processes are the capture, storage,
indexing, retrieval, and archiving of image files.
Discussion:
Agencies and organizations should establish file management procedures
for managing image files for use at a later date.
Question:
Does image processing alter images?
Answer: Yes.
Discussion:
The purpose of image processing is to alter the images in a controlled,
predictable, and repeatable manner. Image processing does not mean
that the input image is overwritten during the process. Forensic
image processing should only be performed on working images.
Guidelines
for Digital Image Processing Standard Operating Procedures
The purpose
of image processing procedures is to apply processing techniques
intended to enhance, restore, compress, and/or analyze digital images.
The success of the processing of digital images is measured against
the four legal tests: reliability, reproducibility, security, and
discovery. To achieve success, standard operating procedures should
be followed. Appendix A is a sample
standard operating procedure.
Guidelines
for Equipment
The agency should
address the following minimum hardware and software equipment requirements.
Hardware:
- Input/capture
device
- Image processing
systems
- Output devices
- Storage/archive
Software:
- Image management
- Image processing
Guidelines for Procedures
Agencies should
establish specific step-by-step procedures for image processing
according to agency requirements using SWGIT guidelines. These procedures
should address the following as a minimum:
- Capture
- Processing
- Storage/archive
- Image management
- Security
- Output
Guidelines for Calibration
If necessary,
agencies should develop calibration procedures specific to their
needs.
Guidelines for Calculations
If necessary,
agencies should develop calculation procedures specific to their
needs.
Guidelines for Limitations
Agencies should
take into consideration agency-specific budget, equipment, management,
and accrediting agency requirements.
Guidelines for Safety
Agencies should
develop safety procedures specific to their needs.
Guidelines for References
Agencies should
maintain its agency-specific documentation, manufacturers' manuals,
and SWGIT guidelines.
Guidelines for Training
Agencies should
document procedures to ensure sufficient training to afford competence
and proficiency with applicable image processing. Refer to the Guidelines
and Recommendations for Training in Imaging Technologies in the
Criminal Justice System at the following:
www.fbi.gov/hq/lab/fsc/backissu/april2002/swgittraining.htm
Appendix A.......Appendix
B.......Appendix C
Top
of the page
|