Submitted to:
Department of Health and Human Services
Assistant Secretary for Planning and Evaluation
Submitted by:
Westat, Inc.
Chapin Hall Center for Children
James Bell Associates
April 4, 2002
This report is available on the Internet at:
http://aspe.hhs.gov/hsp/fampres02
Chapters
List of Tables
Table 4. New Jersey Monthly Foster Care Board Rates (January 1998 to December 1998)
Table 5. New Jersey Average Annual Cost per Placement Type (FY 1997 to FY 2000)
Table 6. Tennessee Monthly Foster Care Board Rates (July 1998 to June 1999)
List of exhibits
Beginning in 1994, the Evaluation of Family Preservation and Reunification Programs was undertaken by Westat, the Chapin Hall Center for Children at the University of Chicago, and James Bell Associates under a contract to the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services (DHHS). The evaluation's purpose was to determine the extent to which family preservation and reunification programs are effective in meeting the basic needs of children, in promoting improved family functioning, and in reducing unnecessary foster care placements. The study was to explore the extent to which these programs have varying degrees of success with different target populations and to compare the costs of treatment cases that received family preservation services to the costs of control cases that received "regular" child welfare or other closely related services.
The cost study design was developed as part of the original evaluation plan in 1995. However, analyses of survey and administrative data gathered through the evaluation found there were no significant differences between the treatment and control groups on foster care placement and other outcome measures of interest. Because of the evaluation findings, a consensus among advisory panel members was that the field would benefit more from a report on how a cost study can be conducted in the child welfare field than pursuing a comparison of costs of services for treatment and control cases in this study.
This document presents our refined methodology for conducting a cost study of family preservation services based on the lessons we learned from the four study sites for which plans were developed. The methodology is designed for this specific evaluation; however, it can be adapted for other research studies requiring cost analyses or for ongoing budgetary and cost analyses efforts in child welfare programs.
The goals of this report are to provide a description of what may be encountered in the child welfare cost data environment and to develop a methodology that includes a comprehensive framework for developing and obtaining service units and unit costs in order to assess the cost of services to families involved in the child welfare system. Although the cost methodology was not applied in this evaluation, within a larger child welfare context, the lessons learned through this work seem more relevant than ever as efforts to hold social service programs accountable are gaining popularity.
The field has acknowledged the need for a cost methodology that can quantify more accurately both the levels of effort and costs expended in the provision of services. The most recent laws passed and initiatives developed in the administration of child welfare focus heavily on accountability and cost effectiveness. These are briefly described in the next section to provide a background for why new methods for tracking and monitoring the true costs of service delivery are now so critical.
[Go To Contents]
In the past decade, child welfare programs all across the country have placed increased emphasis on permanency, safety, and well-being for children and families involved in the child welfare system. Both state and federal laws were enacted that mandate timeframes for permanency decisions for children in foster care.
The Adoption and Safe Families Act (ASFA) included a requirement for DHHS, in consultation with states and experts in the field, to identify a national set of outcome measures that can be used to gauge state and national progress in reaching the goals of child safety, permanency, and well-being for child and family. DHHS finalized rules by which state child welfare programs can be monitored by the established outcome measures. The federal review process has been updated so that DHHS conducts state-by-state reviews for rating the performance of states in operating child protection and child welfare programs.(1)
In addition to the legislation emphasizing outcomes, the federal government has approved waivers of Title IV-E funding limitations in several states. The waivers allow IV-E funding of service programs directed at permanency for children beyond those directly related to a foster care placement. The waivers require evaluations that address success of programs and cost containment. It is hoped that the waiver demonstration projects achieve better outcomes for children with the same or less money. If cost savings are realized then the savings can be reinvested in other child welfare activities.
In concert with the focus on outcomes and results, federal and state governments continue to seek viable mechanisms by which performance measurement based on client outcomes can be tied to cost effectiveness. The goal is to develop a cost-effective service delivery system that produces the best possible outcomes for children and families. In the current legislative environment, incentives in the form of bonuses are given to states with improved performance in the number of finalized adoptions, child support collections, and TANF successes.
In addition to federal bonus incentives, new federal regulations call for the design of performance-based budgeting systems. Some states have already implemented performance-based budgeting in the planning, developing, and monitoring of state budgets. Performance-based budgeting ties performance-outcome measurement to the funding of programs. This budgetary process first requires the development of a strategic plan that specifies the goals and objectives of service. In a service agency, the goals and objectives should relate directly to successful outcomes for its clientele. Services are defined and developed that are believed to contribute to obtaining those goals and objectives as measured through outcomes. Outcome measures must be quantifiable. The defined client outcomes should be obtainable and reasonable considering available resources.
An agency then develops strategies by which it can deliver the service successfully within its resources and produce good outcomes for its clients. As in any business providing a service, the characteristics of the clients receiving the services are analyzed, the cost of providing services is tracked, and client outcome data are gathered and analyzed.
A complete description of performance-based budgeting requirements and methods is beyond the scope of this report; however, the capability to perform accurate cost analyses of the services provided and an accounting of to whom they were provided is one of its requirements.(2)
Accurate cost analyses require the measurement of how much service each client received (the number of units of service per client) and applying unit costs to obtain accurate service costs that are based on the level of effort expended per client. True service cost cannot be computed for a client until service components (units of service) are discreetly identified and tracked, and the price of each component (unit cost or unit rate) is determined and assessed. Without cost analysis capability based on the units of service completed by individual clients, the performance measurement or the cost effectiveness of delivering services based on the outcomes of a varied and complex client population cannot be accurately determined.
Increasingly, state and local child welfare systems are attempting to measure progress toward specified outcomes. Beyond this, members of state legislatures, facing tightening budgets, are demanding to know the success of the services they are financing. Similarly, program managers undertaking strategic planning efforts require information on the relative costs of services and programs and the tradeoffs involved with financing alternative approaches. In short, the demand for cost data and analysis is growing; yet, those seeking this information quickly run up against a system that remains largely impervious to their efforts. Other than foster care maintenance payments, financial information is not consistently maintained on either a case-specific or service unit-specific basis.(3) As a consequence, little information is available on effective approaches for measuring cost within a child welfare services environment.
Though conceptually simple and widely applied in some fields of health and human service, case-level cost analyses are extremely rare in the child welfare field. This is because historically, child welfare agencies have not been required to manage their systems from the perspective of financial accountability. In addition (and perhaps as a consequence), only a limited number of child welfare service costs are claimed or tied directly to a specific child or family. Often, even if the payment for service can be tied to a child or family, the level of effort expended in terms of time and activities performed in the delivery of the service is not captured.
A unit rate structure, if designed correctly, is the best method for compiling accurate service costs. The tracking of the amount of service each client receives in terms of the number of units of service provided allows for a good estimation of the level of effort expended per client. The cost of service is calculated by multiplying the unit rate times the number of units of service that were received. Some child welfare services provided through contractual agreements are billed to the referring agency using a unit rate structure. Foster care board payments are a good example of this type of contractual agreement. The unit of service is a day of board and room for a child. The facility or type of facility has a contractual rate per day based on the services available at the facility.
Often, however, agencies will use a simple administrative contract, similar to a block grant agreement, that does not require the accounting of service delivery to a specific client. The provider of service is given a lump sum amount to serve either a set number of families or to provide the service until funds are expended. Bills are submitted to the referring agency but do not include a record of the amount of service provided or the identification of the clients who received the services. The results of the services delivered are almost never reported.
In addition, child welfare agencies refer clients to other state and locally funded service agencies for mental health, medical, food, and housing services and often do not ask for verification and documentation of service receipt. Data retrieval is also complicated by the fact that needed information resides in more than one agency's records and case files (e.g., child welfare, social service, mental health, etc.), and it is normal for each agency to maintain its own separate payment system.
Furthermore, child welfare caseworkers do not record their own time — a major cost element — by case or discrete activity type. For many services, agency records do not show if the service delivered to a family involved 10 minutes or 10 hours of child welfare caseworker time. In the sites studied, case documentation did not include agency staff level of effort.
Due to these complexities, most prior cost analysis in this field compared only average costs; for example, the average cost of providing family preservation with the average costs of providing foster care. These two services are among the few in child welfare where cost and service delivery come close to being well documented for a family and child. But even for these services, there are problems in calculating complete service costs and the level of effort per client.
Family preservation services are generally charged to the referring agency using a fixed rate per family using an average treatment period (e.g., six weeks or six months). A family of three is charged the same as a family of six. The actual number of family members who get service and the amount of service hours vary depending on the cooperation of family members. These variations are not tracked in the accounting system and may or may not be documented in the agency's case records.
The cost of foster care room and board as stated above is tracked using a unit rate structure in most states. The child welfare agency is charged for the cost of care using a daily rate based on the level of services provided by the facility providing care to the child. The type of placement and level of care provided range from a foster family home with room and board only to intensive inpatient residential treatment centers that provide therapy, mental health, and medical services as well as room and board. However, there are other undocumented costs in foster care. A child is placed in a facility based on the level of care needed by the child at the time of placement. Over time, the child's needs may change. The child may need more services or may not need the services available at the facility. Reassessment of the child's needs is an ongoing effort in order to match the child to the best placement and rate. Some of the undocumented costs are the worker's time in assessing, arranging, and monitoring the placement; developing foster homes; and court services.
However, to provide a more complete accounting of cost, it is necessary to expand on existing cost analyses and improve accuracy by including a more comprehensive array of services and casework activities into the cost framework. Accompanying the framework, a cost-analysis plan should provide methodologies and systems for capturing not only service costs by case but also the level of effort expended per case in delivering the service. One method, defining a service by its discreet activities and developing units of service and unit rates will be discussed further in the next section.
[Go To Contents]
The following section provides a cost methodology based on the lessons learned at the four study sites and applying cost methodologies from previous cost studies. The primary focus of the study was to assess the impact of family preservation in four sites and reunification services in one site. The family preservation study involved randomly assigning children to treatment and control groups in four sites: Kentucky, New Jersey, Philadelphia, and Tennessee.(4)
In conjunction with survey and administrative data collection, between December 1996 and January 2000, the study team completed a series of telephone interviews and site visits to each of the four family preservation study sites. Information from this work was used to develop site-specific cost data collection plans that both recorded information availability and documented site-specific factors that would have to be taken into account.
One of the greatest challenges was that at two sites, both administrative and financial payment systems changed over the course of the study. The descriptions of each of the four study site's cost data, data sources, and other site-specific issues are discussed more fully in Appendices A-D.
A variety of cost analyses models are available that have been developed for human services. The methodology used for this report was developed under consultation with Ernst and Young LLP. The methodology relies heavily on an ingredients-based approach, formally known as the Resource Cost Model, originally developed by Hartman (1979) and Chambers and Parrish (1983). Other sources for this cost study methodology are from Harkavy and Bond, (1992), Wolff and Helminiak, (1993), and Olds, Henderson, Phelps, Kitzman, and Hanks (1993).
The methodology requires a list of the ingredients and the unit cost of each ingredient. The ingredients that make up costs in child welfare are service components or units of service. A quantity of an activity performed in providing a service defines the unit of service. For example, an hour of counseling, a trip to and from treatment, or a day in foster care is considered a unit of service. A rate or unit cost is established for each component. In the best circumstance, the costs of all service components can be gathered and computed for an individual client.
The sum of those costs produces an estimate of the total cost of serving that client. The advantage of calculating the costs of services by tracking the number of units of service delivered to each client and applying the appropriate unit rate is that the costs are easily summed by type of service or for any particular group of clients included in the cost analysis. Summing the costs of all the service components that apply to that service can produce the total cost of a service, such as family preservation or foster care. Additionally, the tracking of the units of service provided to individual clients allows for the estimation of the level of effort required to provide services to special-interest client groups (e.g., child welfare clients who are substance abusers) when these characteristics of the individual clients are also tracked.
This methodology is organized into six suggested steps that include designing a data collection framework; defining targeted services by service components called units of service; determining how service activity is recorded and tracked; identifying cost data sources such as accounting systems, classification and salary schedules, and unit rate tables; establishing data retrieval methods; and lastly performing cost data analysis.
Each of these steps requires considerable and ongoing input from the study site. For this reason, if data are being collected from more than one site within a state, it is recommended that most of the steps be repeated at each site to verify the cost data elements and data retrieval methods. This exposes any unexpected differences in policies and procedures that affect data collection and analysis. Moreover, it is advisable to complete steps 1 - 5 prior to initiating the collection of administrative data. This is because, depending on the quality of existing data sources, it may be necessary to implement new data collection forms for the study population.
Be aware of any existing plans for changes to service reporting and financial systems at each site. Map out alternative plans of action for data collection in the event these systems do change during the study timeframe. It is necessary to map old systems to new ones so that data remain consistent throughout the study. This section addresses in detail each of the six steps of the cost methodology.
[Go To Contents]
In order to design a cost data collection framework, it is necessary to define clearly the research questions to be addressed. A cost study can be used for several purposes. For instance, it might be conducted for the purpose of:
Clearly defining the understanding to be developed and selecting those services and activities believed to affect the analysis are necessary to refine the data elements to be included in the framework. For example, if cost effectiveness based on outcomes is the goal of the cost analysis, then the relevant services included in the study are those that are most likely to influence the outcomes.
Table 1 lists typical child welfare services and functions that were included in the cost study data collection plan designed for the Evaluation of Family Preservation and Reunification Services. The framework design was based on three expectations: 1) foster care and family preservation services account for the largest percentage of the cost for each case; 2) caseworker labor is a significant component of cost and accounts for differences in the average cost per case; 3) several other key services, other than those associated with family preservation and foster care, are likely to contribute significantly to the average cost per case.
Though applied somewhat differently across sites, three major classes of cost are used to account for public expenditures on child welfare. These are child welfare services provided and paid for by the public child welfare agency, services provided and paid for by an external agency, and indirect costs incurred by the public child welfare agency that support the general operation of the agency such as fringe benefits, administration, and other overhead costs.
| Service Category | Service/Cost Components | Activities and Units of Service | Rate Structure |
|---|---|---|---|
| Family Preservation Services |
|
|
|
| Foster Care Services | A. Maintenance Payments
B. Other Miscellaneous Payments
C. Foster home development
|
A. Days of care per child
B. Per child/family
C. Application
|
A. Rate per day
B. Set amount per family or child
C. Rate per application process
|
| Other Contracted (or Externally Provided) Services |
|
|
|
| Emergency Outlays |
|
|
|
| Child Welfare Agency Casework |
|
|
|
| Child Welfare Agency Indirect Costs |
|
|
|
The framework and all decisions regarding cost inclusion and exclusion criteria must be consistent with the research questions that define what it is that is to be learned. Due to variations in both the structure of child welfare systems and the availability of community services, it is necessary to review the list of targeted services with site staff to determine service availability and the service delivery system. For example, while all four of our study sites used contracted family preservation services, other states deliver this service directly through public child welfare agency staff. In addition, some sites that use contracted family preservation services will have designated family preservation contract monitors where others might use ongoing protective workers to monitor family preservation cases. In either situation, both sets of costs (contract and agency personnel) need to be included in the cost analysis.
Whether the objective of the cost study is to answer a research question involving a comparison of cost between a treatment and a control group, to evaluate cost effectiveness, or simply to determine the cost of service delivery, it is essential to record all service costs consistently. If a service is identified as relevant, it should be included for all cases even if multiple sources must be queried to determine cost. For instance, if psychological assessments are selected as a cost component, all incidents of this service should be included, regardless of whether the assessment is paid for by the child welfare agency, Medicaid, or through private insurance. To collect this information on some cases but not others will bias the information collected. Again, the analysis should be an objective account of all service expenditures included in the cost framework.
There must be a consistent internal logic that drives all decisions about the breadth of the cost components. The public child welfare agency, an external agency, or a combination of the public child welfare agency and external agencies may incur the service costs included in the framework provided in this document. A decision was made in the cost study design for the Evaluation of Family Preservation and Reunification Programs to limit external services to those that are directly related to child welfare outcomes or that duplicate services provided by the public child welfare agency. Excluded are physical health care, income maintenance, and other services and assistance for at-risk families and children that fall outside the public child welfare agency domain.
Following discussions with site staff on the structure of the child welfare agency and data availability, it will be necessary to refine the framework. Final decisions about the framework will also be influenced by three factors:
The foremost consideration should be the extent to which families in the study population use the service. If a majority of the study population is receiving a service — especially if those clients are highly skewed to the treatment or control groups — inclusion of this service cost is essential. This might include alternative home-based services available to the control group as an alternative to family preservation.
A second factor, the amount of resources available to the study team, may also affect data collection efforts. For example, some of the data needed to estimate service costs might have to be collected manually from case records. Needless to say, this is a labor-intensive activity and a significant cost to the study team. The issue that needs to be addressed when resources are scarce is whether to cut the number of sites or the scope of the study. If it is necessary to reduce the number of cost variables due to prohibitive cost of data collection, it is important to do this without introducing bias.
One way to reduce the cost of data collection in a comparative study is to test whether certain services have similar costs for each group of clients. Service variables that are found to have similar costs regardless of the client group or client characteristics can be eliminated from the cost analysis.
Third, due to variations in financial accounting systems, producing case-level and unit-based cost information will be easier for some sites than it will be for others. For example, among our study sites, Tennessee's child welfare agency operated multiple payment systems, each of which would be used to provide a portion of the necessary data. On the other hand, Philadelphia operated a single payment system that was already tied to its administrative database and was able to give us case-specific data (in certain areas) with relative ease.
In all cost studies, decisions need to be made about how to use available resources to produce accurate results. The objective of the cost study can help in these decisions; however, accurate results to meet the objective should not be jeopardized by the limitations in scope or the number of sites.
Finally, a decision must be made about the time period to cover for cost data collection. In comparative cost analysis between treatment and control groups, the cost data collection period should mirror the time period for collecting service and outcome data. If services and outcomes are tracked for 12 months from case selection, then cost data are needed for the same period in order to capture the correct costs paid during that time for the services provided to clients in the study.
It is also important to capture effective dates of all applicable rates during the period covered. In child welfare, as in other fields, payment rates fluctuate over time, even if based only on inflationary rate increases.
In tracking cost of service delivery and in performance-based budgeting, the period covered by cost data collection is often dependent on the budget cycles of the state or federal governments, and these are normally based on fiscal year. States will often require reporting on performance and cost measures for monitoring the budget on a monthly, quarterly, and yearly basis.
[Go To Contents]
Learning how a particular state or site defines its services is a necessary step in analyzing cost. Often similar terminology is used in different sites, but the actual activity components of a service are delivered in different ways and can contribute to a higher or lower cost of service. This becomes important in determining what service activity components are included in staff salaries, in the agency's budget by object codes, or in a rate, fee, or payment for contracted services. As an example, foster care services in one site may include only foster family home and foster group home placements and not residential and institutional care. Other sites may include all out-of-home placement activity, including institutional care, in its definition of foster care services but may also have some placements in institutional care paid for by Medicaid. Foster home development, monitoring, and licensing may be included in the per diem rate by type of facility. For other sites, this function may be administered by a separate agency or by a group of specialized workers with separate funding. The level of care determination and the job of finding the most appropriate home for a child may be handled through a managed care contract and, therefore, be excluded from a public agency's worker activity.
Costs for direct service worker activity are almost always captured as an average cost per client based on an aggregate of workers' total cost to the agency that includes salary, time, benefits, and supporting staff. Knowing whether the cost of service is recorded for an individual client at a unit of service level or only as an average cost per client is critical in comparative cost studies of special-needs client populations or of treatment and control groups. In this cost study, as in most child welfare cost analyses, there would be a mix of average costs per client and those costs that can be obtained at the individual client level.
Knowing what is actually performed in the delivery of a service is also important when developing a list of service activities that are reported in direct service workload time studies or if case-specific service time logs are used in the study. The intake process for handling calls reporting abuse and neglect is an example of the need to define activities by discrete categories. Some sites include all phone calls made to the protective service agency, not distinguishing information and referral calls from calls about a possible abuse and/or neglect incident. Even in an investigation, the activities involved may become indistinguishable from in-home service provision. Many states allow and encourage investigation workers to initiate in-home and support services to the family as quickly as possible to avoid placement.
Variations in the definitions of terminology used in child welfare can often create misconceptions about the data stored in the administrative systems as well. For instance, a case opening for service was defined and recorded in the case management systems differently at all four of the study sites. It is extremely beneficial to ask for systems' documentation. Services may be defined in policy a certain way, but the automated system may process the data using a different definition. For example, in Kentucky a case was opened when an abuse/neglect incident was accepted for investigation and the opening date for service on the system was the report date. However, New Jersey used a multi-tracking system that allowed cases to be opened for service due to family problems or due to abuse and neglect after the investigation. The date opened for ongoing services was discrete from that for the investigation services.
Defining a service by the activities performed by each study site in the provision of that service becomes important in the interpretation of the data and assessing the cost of services. Since child welfare cases have inherent variations in the amount and duration of even the commonly defined services, it is crucial in a comparative cost analysis that the service definitions be clear.
After the cost study services are chosen and defined, it must be determined how the agency records the data necessary to track clients, services, service costs, and outcomes. An important step is to learn and understand the systems the agency uses, either automated or manual, for record keeping and where data on client demographics and characteristics, service provision, and service costs are collected and stored. Along with learning the record-keeping procedures, it is just as important in a cost study to determine how the agency makes payments for services, what mechanisms and systems are used to record payments to external providers and salaries to agency staff.
Understanding how services are funded facilitates efforts to retrieve cost information. For instance, several services, such as substance abuse treatment and mental health services are usually delivered and/or funded by outside agencies. These should be considered for inclusion into the cost analysis because they directly affect child welfare outcomes. However, substance abuse services can be funded by the child welfare agency, through programs delivered by state health or public health departments, Medicaid, private insurance, or through independent grants to community providers. Obviously, data collection for this type of service presents certain challenges that should be considered carefully and resolved as early as possible. This may involve developing cost-estimation procedures that can be applied to hard-to-get data. Cost- estimation procedures for externally delivered services are discussed more fully in section E-3 below.
Due to the fact that a significant amount of cost information may reside outside of the child welfare system, it is necessary to determine the extent to which other agencies will share cost information with the study team. Getting buy-in at the early stages of study design is critical to conducting the cost analysis. For example, New Jersey operates an automated accounting system, called the "child ledger" that tracks a wide range of child welfare services and ties the expenditures directly to individual children. The state's Department of Human Services (DHS), the umbrella agency that includes the child welfare agency, administers the child ledger system. The child ledger is not an online system. It is the product of matches between multiple payment systems and the state's administrative service information system. Child welfare agency staff receives printed reports each month from the system to verify the accuracy of claims. Our child welfare contacts recommended that we extract cost information on study families from the child ledger printouts because they did not have access to the automated matches. Data collection from the monthly child ledger reports is labor intensive.
Had the cost study actually been conducted, it might have been possible that through up-front pre-study arrangements, a cooperative plan between DHS and the Department of Youth and Family Services (DYFS) could have been established whereby DHS could provide the automated payment match files to DYFS routinely. This arrangement could have been made to last only for the study period or on a permanent basis beneficial to both agencies in tracking expenditures and tying service delivery to client outcomes.
Once the range of services has been selected and the source of cost data identified, it is necessary to understand how the local payment systems are structured. States use a variety of financial systems to track service contracts and other child welfare expenditures. From our experience, there are two basic types of automated systems used for tracking contracted services that can supply valuable information for a cost study:
Automated payment systems are the ideal source of cost data because they track case-level expenditures by the number of units of service, the unit rate, and the dates of service. This gives the study team actual case-level data with the quantity of service and the amount spent and, ideally, a data tape from which costs can be sorted and analyzed without having to construct cost estimates as well as a new database to store the estimated data.
In addition to these two automated financial systems, states use a variety of other accounting systems, including manual records or ledgers that can be used to gather case-specific data. For example, Kentucky tracks a variety of contract costs using case-specific vouchers and invoices. In Philadelphia, records for the use of flexible funds are maintained manually on a ledger that identifies case-specific expenditures. Similarly, New Jersey district offices use local bank accounts from which they can draw down flexible funds, and case-specific records are maintained on all expenditures.
In order to gather cost data from any of the financial systems, it is necessary to understand the identifying information used by agencies. Generally, sites employ one of the following: case name, a combination of case name and birth date, the child welfare case number, or the child or parent's Social Security number. It is important to understand the identifying information needed by the finance office in order to retrieve the identifiers during the collection of administrative data.
Child welfare agency personnel budgets are generally maintained by line items that represent the worker's service area or unit (e.g., investigation, foster care). Budgets for each service area — including costs for front line, supervisory, and support staff — are generally based on caseload projections multiplied by a weighted service unit cost based on the unit's staffing structure. The level of detail maintained on agency personnel costs is dependent on the number of staffing functions used by the finance office. During the study period, none of the sites had a concrete methodology for constructing case-specific expenditures of public agency workers. Kentucky for many years depended on an elaborate client-specific service activity reporting system that was tied to the state's cost allocation formula. At the time of the study, this system was no longer being used or maintained. The old social services systems were being dismantled and replaced with the newer case management systems.
Depending on the period of interest to the study, it may also be necessary to learn about the availability of historic data and how cost information is archived. At a minimum, it is important to understand what data are archived and how they are maintained — on an automated file or in hard copy — and the complexity of accessing the information.
Finally, it is also important to determine if the site plans to modify (or fully replace) any of its financial or administrative systems during the course of the study. This occurred in both Tennessee and Kentucky and clearly has several potential ramifications ranging from the use of new case identifiers to changes in the types of services claimed and the claiming system itself. In order to analyze outcomes using administrative data in Kentucky, data elements from the old and the new systems were mapped to each other to maintain consistent definitions of services, and service dates were cross-referenced for an accurate account of the service history for each client. This was no small task, and, therefore, changes in data collection and payment systems during a study period can affect the accuracy of the analysis as well as increase the cost of the study.
In Kentucky, one aspect of the new systems should prove beneficial for later cost studies. That aspect is the tying of the new case management system to the financial payment systems. Whenever the administrative case management system is used as the source for the record of service by the financial payment system, reporting of the provision and cost of services is likely to be more accurate. This is explained in more detail in the next section.
Once the targeted services have been clearly defined and service recording, funding, and payment systems are identified, it is time to review the data sources. This step is important in the understanding of not only what data can be retrieved, but also their accuracy. There may be many possible sources but only a few reliable ones. Additionally, completeness of record is important to any cost analysis.
When case management systems are tied to payment systems, the data are generally more complete and accurate. This is true of foster care payment systems in many states where workers record each placement in the case management system, and these records are used by the payment system to create invoices for payment to a facility or agency by child. These systems are edited against each other daily, weekly, or monthly depending on the billing cycle. Kentucky's new MARS builds its placement records for payment from the case management system, TWIST; Kentucky's SACWIS is described in more detail in Appendix D. This constant editing and the need to get accurate payments to foster parents make them more accurate and complete than systems that are not used for paying bills.
The new State Automated Child Welfare Information System (SACWIS) being developed in many states are designed to be useful for workers to manage their cases in an automated form, thereby making the record keeping less cumbersome. The verdict on the success of this endeavor is still not in. Under this premise, these systems should prove more complete in tracking services to clients by activity components. The older child welfare administrative data systems, while able to provide the total number of clients served in global service categories such as investigation, in-home, and foster care, did not provide the necessary discrete service activities required for unit costs analysis.
In all of the study sites, records on cost and services were available for review in financial payment, administrative data, and contract administration systems. Other records were available on paper reports such as ledgers, invoice and voucher files, case records, and contract files. Asking questions of knowledgeable staff is critical to the understanding of the data. Data sources will either complement, overlap, or duplicate each other. In some sources that duplicate or overlap, the data may not match record for record. It is important to find out why they do not match. The aim is to find the most complete and accurate source of needed data on clients, services, and cost.
Other types of records that are important to review in a cost analysis are rate setting methods, structures, and schedules; cost allocation and budgetary formulation; policies and procedures regarding contractual agreements; staff salary levels and functions; and workload analysis and time studies already performed by the site. The information provided by these records is helpful in understanding the data that are eventually retrieved and used in the cost study.
Once all possible sources have been reviewed for completeness and accuracy, a determination of what sources to use in data retrieval for the cost analysis is made. A helpful tool is a data matrix listing all the cost data sources by the services being studied. The data matrix can include details about the source such as accuracy, completeness, and utilization dates (e.g., the period of operational use of a form or system by a site). It can include important information about how the cost data are stored, in hard copy or on an automated data file. For each source, the unit of recorded cost can be defined by whether the data are recorded by units of service per client, by client, by case, by staff person, by units of service by contract, or by contract amount. A data matrix was prepared for each of the study sites, and these can be found in the descriptions of available cost data in Appendices A - D.
[Go To Contents]
Using the data collection framework as well as information on the agency financial and contracting systems, it is then necessary to review each targeted service with the site to develop a cost-retrieval system. As described above, the child welfare agency will probably be able to produce either case-level expenditures or unit cost information on its own contracted services. Some targeted services may have more than one source of funding (e.g., through contract with the child welfare agency, Medicaid, other public agencies or through community-based organizations). Consequently, all sources must be identified and contacted to determine the best way to identify payments made on behalf of study families.
Turning first to child welfare services contracted by the agency and other externally provided services (categories 1-4 of the data collection framework presented earlier in Table 1), we have listed a series of overriding questions that should be reviewed with site financial and administrative personnel.
For each contracted service determine:
For those service costs that cannot be collected on a case-specific basis:
The following tables present the service categories from the original data collection framework developed for the Evaluation of Family Preservation and Reunification Programs (contracted family preservation, foster care, other contracted services, directly delivered agency services, and indirect costs). These tables present the relevant issues to review with agency personnel for each service or cost component and may be adapted to other service models.
The following table summarizes issues to be explored with regard to contracted family preservation services. If public agency staff directly provide family preservation services, refer to the section "Directly Provided Services" below that offers several options to estimate these costs.
| Service Category | Issues To Explore |
|---|---|
| Family Preservation Services |
|
For the Evaluation of Family Preservation and Reunification Programs, family preservation was a contracted service at all four of the study sites. At each site, service unit rates included both therapy and administrative costs. The contracts were based on a set cost per slot, regardless of the actual length or intensity of the intervention. The service unit costs for the intervention were $2,600 in Tennessee, $4,170 in Kentucky, $4,300 in New Jersey, and $4,500 in Philadelphia (1997 rates). In addition, some family preservation models include the potential use of "booster" or follow-up interventions. These interventions may be reimbursed at a lower rate, as the average service duration is designed to be shorter.
None of the sites maintained family preservation expenditures on a client-level payment system. While unit cost information was available from contract management systems, service utilization information would need to be gathered from the study's administrative database or from one of the sources discussed in section 3 below.
Table 3 summarizes issues to be explored with respect to contracted foster care services.
| Service Category | Issues To Be Explored |
|---|---|
Foster Care Services |
Review the following data collection issues for each level of care (e.g.,
foster home, group home, institution, etc.):
|
As discussed above, case-level foster care maintenance payments are nearly always available from child welfare agencies. However, it is important to review the availability of cost information for each type of placement as a state may use more than one payment system to track maintenance payments. For example, Tennessee maintains case-level expenditure data on all types of placements except state-operated group homes. These facilities do not claim on a case-level basis and would require the matching of client-level service utilization information with per diem maintenance rates.
States use a variety of standardized payment rates for each type of placement and, in some cases, negotiate individual rates with group and residential facilities. For instance, New Jersey has developed a payment matrix to reimburse foster homes that correspond to both the child's age and special physical and emotional needs, see Table 4 below.
| Age | Step 0 | Step 1 | Step 2 | Step 3 | Step 5 (Does not correspond to age) |
|---|---|---|---|---|---|
| 0 thru 5 | $294 | $334 | $369 | $409 | $982 Medically Fragile |
| 6 thru 9 | $312 | $352 | $387 | $427 | $1,103 HIV Asymptomatic |
| 10 thru 12 | $346 | $386 | $421 | $461 | $1,344 HIV Symptomatic |
| 13 and over | $368 | $408 | $443 | $483 |
These monthly payments do not include clothing or other special expenditures. In addition, New Jersey does not use standardized rates for group homes or residential or shelter care. Each provider's contract is negotiated separately. For the purpose of the Evaluation of Family Preservation and Reunification Programs, New Jersey estimated the average annual cost per child in each type of placement (Table 5) including the congregate placements by dividing its total annual expenditure by type of placement by the total number of children served in the placement.
| Fiscal Year | Foster Care | Group Home | Residential | Shelter |
|---|---|---|---|---|
| 1997 | $6,799 | $47,270 | $62,871 | $24,643 |
| 1998 | $6,528 | $53,567 | $65,276 | $26,985 |
| 1999 | $7,169 | $54,416 | $71,555 | $25,339 |
| 2000 | $8,139 | $54,615 | $77,791 | $26,321 |
The foster care board rates for the Department of Children's Services (DCS) in Tennessee is presented in Table 6 below. Each placement is made with an Individual Child Placement Contract signed by the child's case manager and the DCS-approved family foster home. The special circumstances monthly board rates are available upon approval of the Residential Case Management Team for the child with unique needs due to medical problems, behavioral problems, or a child who requires extraordinary supervision or requires extra care due to physical, emotional, or mental handicaps. Foster homes that care for children who are moderately, severely, or profoundly mentally retarded receives special rates beginning at $600. Additionally, Tennessee has separate daily rates for foster group homes, residential facilities, camps, and therapeutic and contract foster homes.
| Age of Child | Regular Board Rate* | Special Circumstances Board Rate* |
|---|---|---|
| 0 through 2 years | $336 (includes clothing allowance) |
$513 ( includes clothing allowance |
| 3 through 4 years | $238 | $383 |
| 5 through 12 years | $262 | $391 |
| 13 years and above | $385 | $517 |
| * Added to these rates is $25 per child per month for the care of sibling groups of three or more in the same foster home. Sibling groups include half-siblings and adoptive siblings. | ||
Ideally, payment records for each child in the study can be reviewed to determine the amount of money paid to each home or facility that cared for a child during the study period. If these records are not available, it may be possible to develop a reasonably good estimate of maintenance costs by using administrative data on the child's placement history. In all four sites, at least one automated system tracked placements by entry and exit dates by type of facility. To develop child-level estimates, the number of days in care by facility type could be calculated using the entry and exit dates and then matched with agency rate schedules to calculate the cost for each placement. Payment rate variations based on the child's age and special needs will need to be factored in as well. It is also important to ask for the agency's rate schedules for the appropriate periods in order to apply the rate in effect at the time of each placement. If rates vary both over time and by facility based on negotiations that factor in facility services or the child's needs, as in New Jersey, average rates by type of facility and needs addressed can be calculated over the time period of the study. The average rate over time is not as exact as established standard rates, but it provides a good estimation of the cost of foster care by type of care.
Table 7 lists the primary issues to be considered when collecting cost information on the full range of contracted or externally delivered support services selected for the analysis.
| Service Category | Issues to Explore |
|---|---|
Other Contracted or Externally Delivered Services (e.g., respite care, homemaker services, day care, substance abuse treatment, psychological or developmental assessments, counseling and mental health, parent training) |
For each targeted service, determine:
|
In general, the study team found that it was easier to locate and isolate service-unit costs for contracted services than it was to determine client utilization of those services. Where payment for contracted services was made on a client-specific level through an automated financial management system, services utilization and payments were more easily obtainable by case. Problems arise because the payment mechanisms for the various contracted services are not consistent and are generally contract or agency specific. As an example, day care services in all four sites studied were tracked and payment made from multiple systems depending on which state agency or funding stream paid for the service. Protective day care was generally paid for by the child welfare agency, but if a family was TANF eligible, the day care would be paid by that separate agency using a different payment system. Another example is mental health services, which most often are paid for by Medicaid through the health services agency. However, for those not Title XIX eligible, child welfare may pay using a different payment method. In Kentucky, substance abuse treatment services for children are paid through the child welfare system and are tracked through the automated financial payment system, on a manual payment ledger, or through paper invoices. Substance abuse treatment for adults is paid for by the mental health agency through the Medicaid system. Even in the same agency, payment for services can be made and tracked differently depending on the contract negotiated. As in the Kentucky example above, payment for services can be tracked on an automated system, in a manual ledger, or on invoices and vouchers depending on which contract provides the service.
In none of the sites were workers required to enter a comprehensive list of services received from outside providers in either the automated case management systems or case records. Workers may have noted a service referral, but rarely was there a clear indication of service receipt. Sources for determining client utilization, include one (or a combination) of the following:
The strengths and weaknesses of each of these options are discussed below.
a. Agency Automated Service Files
The most clear and concise source of contracted services information is the automated financial payment systems that record service utilization and cost at the client/case-specific level. When these systems were used for payment, the client was identified; the service was identified; and the number of units of service was recorded along with the service start and end dates, the unit rate, and the total amount paid. Philadelphia's child welfare payment system and the Kentucky STARS and MARS were examples of this type of payment system. As previously discussed, however, these systems do not contain the full complement of services provided to the case. Because of the incompleteness, other sources have to be identified. Philadelphia's child welfare system is very reliant on community referrals, and thus nearly all cost data for support services must be retrieved from outside providers.
The first source of data to be reviewed for service utilization information should be the child welfare agency's own automated case management system. Most child welfare case management systems have a service screen where workers can enter information on service referrals. It should be noted that at our study sites, workers used these screens for service referral information, that is, as a record of service recommendations. Other states may use similar screens for information on service receipt information, and they might include information on service dates. System administrators should be queried about the purpose, consistent use, and accuracy of the data entered by workers using the automated case management system. In addition, the data comprehensiveness and validity should be reviewed, as front-line workers may not be required to enter certain data fields.
Provided the automated case management systems are found to contain reliable and useful information on case-level service utilization, there are various ways of using the information to estimate service costs. Two alternative methods are described below.
The first method would involve retrieving case-level cost data on each service from the service-providing agencies listed by the worker. The major drawback to this approach is that it will necessitate submitting client names and approximate dates of service to a range of service providers who would then supply the requested cost information. The process could be costly and time consuming with additional concerns about client confidentiality.
An alternative approach would involve applying average or estimated service costs to the service information contained in the automated case management system. Average service costs would be based on information on actual service duration (if available) or on average service duration collected from agency contracting records, case worker interviews, and/or information gained from the caseworker service logs collected on a sample of study families (discussed below). This utilization data would be applied to service unit costs provided by the child welfare agency and all participating community service providers.
If information on actual service duration is not available from case records or automated files, estimates of average service duration can be developed. This can be done by reviewing agency contracting records and generating an average number of service units received per child or family. If there are wide variations in service duration among cases, identifying client characteristics that correlate to different lengths of service duration could provide a more accurate average number of units of service per type of case. For example, home visiting services may, on average, be delivered longer to first-time mothers than to mothers with other children in the home. Again, the use of information gathered from the caseworker service logs conducted on a sample of families can be used to construct profiles of different groups. This approach, however, should be thoroughly tested for accuracy and may be less than accurate when applied to subgroups such as family preservation since these type of cases are unlikely to be average.
To develop average unit cost information, it would be necessary to know the range of typical expenditures made for a service and the proportion of payments made at various rates. For instance, an hour of counseling may range from $60 paid for by Medicaid to $100 paid for by the child welfare agency. If 80 percent of the study population is found to be Medicaid eligible, a reasonable average cost for 1 hour of counseling is $68 — ($60 X 80%) + ($100 X 20%) = $68. Unit rates for certain services are sometimes standardized and those are readily available, but if they are not, a rate can often be found in the signed contract for each provider. Therefore, if unit rates vary by provider and the provider of the service is known, the rate normally charged may be found in the contract administration files.
If neither agency-automated records nor agency case records provide a reliable account of service utilization, then cost estimates become less reliable. If this is the case, it may be possible to verify service utilization by contacting study families or, by submitting information on service referrals to the service-providing agencies for them to verify service receipt. Otherwise, other sources of information on service utilization should be explored.
b. Vouchers, Invoices, Ledgers
When voucher and invoice systems were used, they were often client specific and provided a good accounting of the services provided to a client with the cost of the service. The number of units of service provided is sometimes available on the invoice. The drawback was that they are manually stored by payment date, not service date, making it difficult to match back to a referral recorded in the case folder. The client is often identified not by a client number used in an automated case management system, but with a Social Security number. Additionally, leafing through files of paper vouchers and invoices would be very labor intensive. Computer-generated or manual payment ledgers contain a good record of service utilization and cost; however, these sources were also stored by payment month, not service month, and are also very labor intensive to find data for a specific child or family member.
Case records contain handwritten contact sheets listing the worker's ongoing interaction with the case, computer printouts from the agency's administrative database, and several forms that provide demographic and case history information. Information on service referrals is noted in a variety of forms as well as in the case notes. Site contacts can help to limit the time involved with case record reviews by identifying the most relevant forms in the records. An example of such a form from New Jersey is available in Appendix E. Even with this assistance, case-record reviews can still be labor intensive and costly to undertake. (It took the evaluation study staff an average of 1 to 2 hours to review each record at two of the study sites.)
While labor intensive to review, case records were found to contain more information about service receipt than the automated administrative data files, including some supporting documentation from community agencies. However, like the administrative data files, there was virtually no information on the length of a service intervention or the number of service units used. If case-record review becomes a serious option, researchers are strongly encouraged to first review a sample of records to assess the extent to which they can supply the needed information.
Estimating service costs from information contained in case records would involve applying the same methods discussed under agency automated administrative data files.
If existing documentation is found to be inadequate for estimation purposes, then the best way to gather a comprehensive and accurate record of service receipt would be to have caseworkers complete service logs that record the full range of services received by study families. At a minimum, the following information should be included on a data collection form: type and description of contracted (or community-based and funded) service received, date(s) of service provision, and length of intervention and/or number of service units received. The forms would be placed in the family's case record upon selection into the study. Because this would be an additional record-keeping requirement for frontline staff, case records should be flagged and have explanatory notes attached to assist with data collection. If the sample size is adequate, the information could also be used to develop profiles of families that use different services in different amounts.
e. Contract Agency Service Reports
Another source of service information might come from the service contractors themselves. There may be instances where the public agency relies on a contract provider to deliver intensive case management services (such as family preservation services) and requires that the provider submit detailed reports about service receipt. This was the case in Philadelphia. Two types of service providers, family preservation workers and the city's less intensive in-home case management providers (Services to Children in their Own Homes - SCOH) are required to submit detailed service histories for all families served. While the study team did not review case records in Philadelphia, the city's Family Preservation Coordinator strongly urged the study team to use these service reports as the primary source of service utilization data.
In Philadelphia, family preservation and SCOH provider reports supply a variety of information including names and often addresses of service providers with whom the client worked during the family preservation or SCOH intervention. Family preservation reports also include provider telephone numbers and dates of service receipt. While the reports do not supply any cost information and would require follow-up contact with each agency, they supply the most detailed account of actual service history found by the study team at any of the study sites. The main drawback to this source is that the reports will only cover the period that the case was in treatment. Therefore, these reports cannot be relied upon as the sole source for service information. If this source of information is available and selected for review, it would be important to determine how consistently these reports are maintained across service providers.
At all of the study sites, Medicaid emerged as a key source of information, as this funding stream was used to fund a range of mental health and substance abuse services that the study team had originally identified as being central to the cost-analysis framework. State and local Medicaid systems may be able to provide case-level expenditures made on behalf of study families.
It would be necessary to gather Medicaid identifiers on all study cases. It may be possible to retrieve Medicaid identifiers directly from the child welfare agency. For example, Kentucky's child welfare administrative files interface with the Medicaid and public assistance files. Otherwise, case names and birth dates would be submitted to the Medicaid office so that system administrators can conduct a special data run to match study names with Medicaid numbers.
One factor that needs to be taken into account is that case-level expenditures may not always be available from state agencies. With the conversion of state Medicaid systems to managed care, some states are now paying providers capitated rates that prevent an analysis of individual costs. If this is the case, service and service cost data may need to be gathered directly from the managed care provider.
States, and often local jurisdictions within states, establish individual policies for the expenditure of flexible funds or emergency outlays made to families. As described earlier, in general, the study team found that states maintain records of these expenditures on a case-level basis. Table 8 lists common expenditures made from these funds and provides some issues to explore to devise a data collection system.
| Service Category | Issues To Explore |
|---|---|
| Emergency Outlays (e.g., food, shelter, transportation, and other concrete needs) |
|
States use a variety of payment tracking systems for emergency outlays and other flexible funds. For instance, sites may use handwritten ledgers or automated payment tracking systems. There may be payment receipts or vouchers in case files. It would be necessary to review data sources for each expenditure because sites may separate some costs into separate accounts such as a transportation voucher system. In addition, many states maintain separate accounts for emergency expenditures for child protective cases versus children in foster care.
Producing case-specific cost estimates of agency casework is perhaps the greatest challenge of a cost study in child welfare. Traditionally, state and local finance offices do not track worker time by family served. Although one of the study sites, Kentucky, designed a new case-specific reporting system for its administrative files, the system was not fully functional during the study period. Without such a tracking system in place, our four study sites were reluctant to conduct cost analyses of directly provided services beyond dividing total annual expenditures for service units (e.g., investigation, foster care case management), by total caseload served. This method generates an annual, average cost-per-case that cannot be used to distinguish differences between the treatment and control groups in service needs and intensity of service activity.
In order to calculate case-specific expenditures for directly provided casework services, it is necessary to first determine both the amount of time workers dedicate to individual case activities and the value of worker labor that would be applied to these activities. To establish labor costs, the study team must understand the range of budget lines corresponding to casework activities. The team must also determine whether sites can generate an hourly rate of labor for each service unit. If so, there is a question of whether the unit includes the labor costs of frontline, supervisory and support staff. In many cases, service unit budgets for directly provided services are based on average annual caseloads (e.g., 20 cases per caseworker and established staffing patterns (e.g., one supervisor and administrative assistant to five caseworkers). These issues will need to be understood prior to data collection in order to determine appropriate units of service information to collect. Alternatively, if personnel costs within different service units do not differ significantly from one another, a standard hourly rate of casework labor (that includes costs of supervisory and support staff) could be generated and applied to all casework activity.
Once the availability of labor unit costs is established, it is necessary to collect or estimate casework time devoted to study families. Several pieces of information are needed to calculate worker time spent on a case. First, standard service units must be developed that can be tracked (e.g., casework activities related to in-home service models or activities related to foster care placement). Second, one must estimate how much time, at least on average, it takes workers to provide the service. Third, the number of times that the service was delivered must be determined. A discussion of data sources follows.
One method to collect actual, case-specific information would be for caseworkers to complete case-specific service or activity logs for each study family. The service logs could be designed to capture both the type of service delivered and the time spent on each activity. Activities involved with each designated service could be developed from worker focus groups or from any workload analysis previously done by the site. Relatively simple data collection forms could be inserted into the family's case record after case selection into the study and completed by caseworkers each time they work on the case. (A sample data collection form for investigative workers is shown in Exhibit 1.) If a case is in more than one child welfare unit simultaneously (e.g., the family is within the investigation unit and one of the children is in foster care), than separate time sheets would need to be maintained in each unit and compiled at the time of data analysis. As with contract services logs, because this is a new record-keeping system requirement, case records should be flagged and have explanatory notes to assist with data collection. If this method is used, it is critical that a validation system be implemented such as random supervisory reviews and/or conducting cross checks of worker logs with case records.
An alternative approach would be to have workers complete a checklist of activities delivered to families each month. Through the use of worker focus groups described below, the study team could generate average timeframes for each activity or service unit.
If sites are unwilling to have workers complete service logs on all study families, forms could be completed on a representative sample of cases. Information on actual service delivery could be used to develop average service duration and applied to actual casework activities identified from a full case record review described below.
| Case Name____________________________________ Date
Case Opened in Agency __/__/____
Case ID_______________________________________ Date Case Closed in Agency __/__/____ Caseworker Name______________________________ Date Case Opened in Unit __/__/____ Activity Period_________________________________ Date Case Closed in Unit __/__/____ Directions: Indicate time spent on each activity in minutes. |
||||||||||||||||
| Type of Activity | Date of Activity by Time Spent on Each Activity | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Month:_____ | 1/ 16 | 2/ 17 | 3/ 18 | 4/ 19 | 5/ 20 | 6/ 21 | 7/ 22 | 8/ 23 | 9/ 24 | 10/ 25 | 11/ 26 | 12/ 27 | 13/ 28 | 14/ 29 | 15/ 30 | 31 |
| Investigation | ||||||||||||||||
| Telephone with family | ||||||||||||||||
| Telephone with collateral contacts | ||||||||||||||||
| Family office visits | ||||||||||||||||
| Travel time | ||||||||||||||||
| In-home assessments | ||||||||||||||||
| Paperwork | ||||||||||||||||
| Court time | ||||||||||||||||
| Supervisory/Case conferencing | ||||||||||||||||
| Other | ||||||||||||||||
b. Use of Existing Workload Standards Analysis
One source of information on casework activity that should be considered is a state's own analysis of workload standards. These projects produce information on both standard service units and service durations, based on casework observation. For example, New Jersey had conducted such an analysis in 1991 for the purpose of determining appropriate caseloads for its case managers. Using a random sample of 425 cases selected from field offices across the state, case workers were asked to record the amount of time required for various activities conducted on behalf of families at various risk levels and at various stages of the child welfare continuum (pre-custody, ongoing/in-home services, custody cases). Once baseline data were gathered, a panel of local and national experts led by the Child Welfare League of America worked to establish case practice standards and duration of individual casework activities.
The casework standards resulting from the project include a description of activities and frequency and duration of activities by case type (see Appendix F for examples of service descriptions). Information generated by a workload standards project can be applied to other sources of cost information to generate estimates of directly provided services. Specifically, the service descriptions and information on service duration can be used in conjunction with other data sources that supply information on the frequency of service delivery (e.g., the automated case management system or case records.)
Information from the workload standards project can be useful, even if the specific findings need to be updated. Components of the analysis can be used for discussions with caseworkers and supervisors as a basis on which to build new estimates of direct service costs. For example, New Jersey's workload analysis specified all of the activities that make up foster care "placement" and also recommended that the bundle of activities that make up placement should take 270 minutes. This information could be used in caseworker focus groups (described below) to devise discrete casework activities and establish updated service duration.
c. Automated Case Management System
Another source of administrative data that should be considered is the site's automated case management system. These systems generally supply information on the number of CPS referrals, investigations of abuse and neglect, the number and length of foster care placements, and the length of time the case was opened in the agency with case opening and closing dates. In short, administrative data files can provide information on the frequency of many of the major bundles of direct service activities provided to a case (e.g., number of investigations, number of foster care placements). However, because this data source does not supply information on the number of worker hours dedicated to each service, information from the case management system would have to be combined with other sources of information discussed in this section.
The major benefits of this data source are its widescale availability and the fact that it is easy to access. Like other sources of data, however, its reliability and thoroughness should be reviewed with site personnel.
In addition, as a component of the case management system, some states have begun to operate automated visitation systems designed to ensure compliance with mandated visitation timeframes. Philadelphia operates such a system that requires workers to record the date and type of home or office visits conducted for each child welfare case that has been opened for service. These systems do not record the time involved with each visit (or any other activities associated with the case), but they offer an automated record of actual casework activity that can be combined with other sources of information to generate case-specific cost information. If the study team chooses to create profiles of families based on risk factors and service needs, information from the automated visitation system could also be used to help classify families based on the number of visits or amount of casework contact recorded by the visitation system. The availability and accuracy of this and any other automated tracking systems of actual casework activity should be explored with site personnel.
Client case records are another source of actual case-specific data. Casework entries on contact sheets should be identified as they can provide detailed information about the type and purpose of casework contact. For example, they describe home visits and phone calls to families and other concerned parties, service referrals and court activity.
Again, reviewing case records is very labor intensive, typically requiring an average of 1 to 2 hours per record. Consequently, prior to choosing to conduct a case record review, it is important to both review a sample of records to verify the amount of information on casework activities they include and determine the best way to abstract needed data. This is important because at one of the study sites, where investigative and ongoing protective service caseloads were very high, case record contact sheets contained relatively few detailed entries. Whether the lack of information was due to limited time spent with each case or a lack of emphasis on recording casework activities was unclear.
The most efficient way to conduct case record reviews would be to develop standardized data collection forms. These forms would include a list of casework activities by service function (e.g., activities involved with investigation) that are developed by an agency's own workload standards project or by worker focus groups. Reviewers would then fill in the relevant cells of the table with the dates of these activities. If collecting data at multiple sites, the casework activities developed at each site should be comparable across sites.
If any of the above data sources are unavailable, intensive interviews with worker focus groups can be used to create standardized service units and generate estimates of service duration. Information gathered from focus groups can be used to supplement other data sources described above. For example, while an automated case management system will provide a discrete list of services or case management functions and can supply information on how many times each service occurred, it does not provide information on the length of time it took to deliver each service. Worker focus groups can be used to develop estimates of service duration.
There are several issues to keep in mind when using focus groups to define discrete casework services. First, as discussed in the initial steps, in order to facilitate data collection, it is important to target a discrete number of casework activities based on their relevance to the study design. Second, assuming that hourly labor rates differ across child welfare service units, it is important to keep the service categories within a casework unit to facilitate establishing a labor cost (or identify the difference in service duration if it is conducted in different units). To do this, researchers must become familiar with the agency's service delivery system — from screening to case closure. Using a list of casework activities, determine which agency worker is responsible for each task. Third, if combining this activity with other data sources, services must be defined in such a way that they can be matched with information contained in those other sources. For example, if case records are to be used as the source for information on actual case activity, it will be necessary to discuss how each service is defined and recorded in the records. Finally, focus on the development of service units that are likely to be provided in different amounts to the treatment and control groups. For example, casework activity involved with monitoring family preservation contracts or delivering the service directly.
Defining some standardized service units will be easier than others. For instance, foster care placement activities represent a relatively straightforward service. Broad placement activities might include:
Establishing discrete services within ongoing case management activities, including ongoing protective services and out-of-home case management, will require detailed discussions with supervisory and line staff. Every effort should be made to create categories that account for the majority of casework time.
Worker focus groups can also be used to help estimate the average amount of time spent on providing each service unit. If time estimates vary widely, it may be necessary to subdivide the service unit. For example, the counseling needs will likely differ significantly between a sex abuse case and a neglect case. Site officials and front line staff should be able to suggest classification systems. The result of this effort would be a list of key casework service units and the average number worker hours associated with each unit. Using data on the cost of an hour of worker time, a unit cost for each service unit would be established.
Indirect costs include an agency's overhead expenses based on costs such as general administration, facilities and equipment, consumable supplies, and staff training. It also includes fringe benefit costs for medical insurance, pension, paid holidays, and paid leave. We found that site finance staff can readily provide researchers with the agency's indirect rate. The rate would then be applied to all appropriate cost elements.
[Go To Contents]
Once data become available and are collected, the cost methodology formulas for calculating the cost of serving clients in the study population can be applied. The costs of each service component, activity, and unit of service can be summed by client, by service, or by any particular group of clients or services. The summation of costs is dependent on the purpose of the cost analysis and the level of aggregation required in answering the questions posed by the study. Due to the range of estimation procedures necessary to conduct a cost analysis in child welfare, it is strongly recommended that procedures for validating information be developed. For example, worker logs and case records could be cross-checked to validate activities listed in the logs. Preliminary findings on the costs of both the treatment and control cases should be shared with site officials to discuss the reliability of the findings. If costs for particular cases appear to be out of line, workers could be interviewed to verify data collection procedures. In addition, meetings between the research team and program staff should take place on a periodic basis relevant to the study period to re-examine the list of targeted services and make any necessary changes if new services have been introduced to the area.
The purpose of conducting a cost analysis in an experimental design study is to determine the differences in the costs of services provided to the treatment and control groups. The validity and breadth of the data, and the accuracy with which the cost information can address the research questions of interest determines the strength of the analysis.
This paper described a series of useful sources of case-level data as well as several estimation methods that can be used to develop other service costs. The paper outlined six interrelated steps that can act as a guide to conducting a cost analysis of child welfare interventions of interest.
Historically, child welfare agencies have not been required to manage their systems from the perspective of financial accountability. To a large extent, child welfare agencies have operated program management and financial systems separately. Only recently have states and local jurisdictions attempted to integrate these systems through, for example, introducing managed care concepts into the financing and delivery of child welfare services. Moreover, many other states and localities are interested in applying these methods within their child welfare systems. Yet these efforts can be stymied by the lack of case-specific and service-specific cost data from which to build capitated rate structures. There is a growing realization that with a goal of improving child welfare systems, program and financial management must become better integrated. Means of deriving accurate cost information on services delivered to clients is an essential element of this reform. The reforms must include detailed systems of accountability for both public agency direct services and contract-provided services.
As local, state, and federal child welfare laws, initiatives, waivers, budgets, and clients demand more accountability for client outcomes and cost effectiveness of services to children and their families, the service providers, both public and private, are moved to reform the systems for accountability. Methods for improvements in recording the data necessary for documenting performance, client outcomes, cost of services, cost containment and effectiveness of services will be valuable. System reform efforts to better integrate child welfare management and financial systems will rely on improved means of deriving accurate costs of services and the outcomes to families served. This paper offers a variety of methods to establish such costs for a wide range of child welfare services and thus can help inform these efforts.
[Go To Contents]
Chambers, Jay G. and William T. Hartman. (April 1981). "A Cost-Based Approach to the Funding of Educational Programs: An Application to Special Education." Institute for Research on Educational Finance and Governance, School of Education, Stanford University.
Chambers, Jay G. and Thomas B. Parrish. (December 1983). "The Development of a Resource Cost Model Funding Base for Education Finance in Illinois" for the Illinois State Board of Education. Final Report of the AEFP School Finance Study for the State of Illinois.
Harkavy, Oscar and James T. Bond. (1992). Fair Start for Children: Lessons Learned from Seven Demonstration Projects. Chap. 10--"Program Operations: Time Allocation and Cost Analysis." Edited by Mary Larner, Robert Halpern, and Oscar Harkavy. Yale University Press.
Moore, Mary T., E. William Strang, Myron Schwartz, and Mark Braddock. (1988). "Patterns in Special Education Service Delivery and Cost." For the Department of Education. Decision Resources Corporation.
Newman, Murray A. (1975). "A Child Welfare Units of Service and Time Management System." Texas Department of Human Services.
Olds, David L., Ph.D., Charles R. Henderson, Jr., Charles Phelps, Ph.D., Harriet Kitzman, Ph.D., and Carole Hanks, DrPh. (1993). "Effect of Prenatal and Infancy Nurse Home Visitation on Government Spending." Medical Care, Vol. 31, No. 2, pp. 155 -174. J. B. Lippincott Company.
Wolff, Nancy and Thomas W. Helminiak. (1993). "The Anatomy of Cost Estimates--The "Other" Outcome." Advances in Health Economics and Health Services Research, Vol. 14, pp. 159 -180. JAI Press Inc.
Family Preservation Evaluations with Cost Analysis:
Bergquist, Szwejda and Pope. (March 1993). Evaluation of Michigan's Families First Program Summary Report.
Citizens for Missouri's Children. (January 1989) Where's My Home? A Study of Missouri's Children in Out-of-Home Placement.
Fondacaro and Tighe. (Spring 1990). An Evaluation of Intensive Family-Based Services, Final Report [Vermont].
Homer, Bass and Cunningham. (December 1991). Family Preservation in Tennessee: The Home Ties Intervention, Select Findings from Operation October 1989 - June 1991.
Maryland Department of Human Resources. (1987). Intensive Family Preservation Services.
McDonald and Associates. (May 1990). Evaluation of AB 1562 In-Home Care Demonstration Projects Volume I: Final Report [California].
Showell. (May 1985). 1983-85 Biennial Report on Intensive Family Services [in Showell, Hartley and Allen, Outcomes of Oregon's Family Therapy Programs (1987)].
Texas Department of Human Services. (September 1986). Impact Evaluation Report: Family-Centered, Home-Based Intervention for Protective Service Clients Project.
[Go To Contents]
1. http://www.acf.hhs.gov/programs/opre/fy2002b.htm.
2. The Guide to Performance Measurement for State Agencies by the Texas State Auditor's Office, Legislative Budget board, and the Governor's Office of Budget and Planning was the resource used in describing the performance based budgeting model concepts.
3. Funding streams related to Federal title IV-E foster care maintenance are income-eligibility tested. For cost claiming purposes, case-specific data must be maintained and submitted by States.
4. The fifth site, the New York City HomeRebuilders project, delivered family reunification and permanency services rather than family preservation. In addition to significant differences in service design, another difference between New York and the other study sites was that the HomeRebuilders evaluation was conducted retrospectively — assignment to experimental and control groups and the intensive phase of the demonstration were complete when data collection began under this evaluation.
To obtain a printed copy of this report, send or fax the title and your mailing information to:
Human Services Policy, Room 404EFax: (202) 690-6562
Home Pages:
Evaluation of Family Preservation and
Reunification Programs
Human Services Policy
(HSP)
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services
(HHS)
Last updated: July 10, 2002