Since Massachusetts community action agencies do not exist in a world by themselves, in one way or another they were bound to be affected by "The Digital Revolution." Yet it seems clear that one factor above all others has been the driving force behind the expansion of computer technology at Massachusetts CAAs since the 1980s: the demand for data on services and clients. And this demand mainly has come from the federal and state agencies that fund programs operated by CAAs.
Although most Massachusetts CAAs operate some of the same federal or state programs such as Head Start, child care, fuel assistance, energy conservation, and housing services, each of these agencies also has some combination of other programs and services. The list of the programs that exist in some CAAs across the state is a very long one: it includes programs for the elderly, the WIC nutrition program, family planning services, employment and skills training, information and referral, and homelessness prevention, to name just a few. Most of these programs are funded through federal, state, or municipal monies. And some individual programs have multiple funding sources. (Click here to view a sidebar showing a sampling of the wide range of programs operated by Massachusetts CAAs.)
In fact, Massachusetts CAAs probably stand out from other nonprofit human services providers in the state in one sense: CAAs probably have more discrete funding sources spread across more governmental entities than do other human services organizations in the state.
Of course, CAAs need to report to the public sector agencies that fund each of these programs, providing data on clients served and the services rendered. As the number of programs and funding sources for CAAs has increased over the past decade or so, so has the data collection and reporting burden.
Unfortunately, except through anecdotal information, we have no evidence to show that over the last 10 or 15 years the amount of client and services data that CAAs must report for the programs that they have operated over that timespan has changed. No study or survey can be found that describes the data reporting requirements for the programs that existed at CAAs in the mid or late 1980s to offer a basis for comparison.
However, it would not be surprising if the demand for data on CAA programs and clients has increased. Testimony offered by longer-term CAA staff who were asked about this topic during site visits would indicate that it has. Moreover, the revolution in computer technology has made it possible to regularly collect, compile, store, transmit, and analyze data in ways that simply were not practical 15 years ago. If nothing else, the capabilities and promise of the new technology have created greater expectations in our society, generally, and within government agencies, particularly, about what is now possible. These expectations—justifiably in many cases—have become assumptions. And these assumptions have been incorporated into the data reporting requirements that CAAs and other human services providers must meet in order to hold onto their contracts and programs.
Massachusetts community action agencies—like other nonprofit human service agencies—have responded to these expectations and requirements by developing their own information technology infrastructures. In most instances, we found that although computer technology has become widespread at these agencies in a relatively short period, the process of development has been an uneven one within individual agencies and has been driven by the needs and requirements of particular programs and their federal or state funding sources. Thus, most CAAs have reacted to these data demands program by program, and they have developed information technology infrastructures that, until at least recently, largely have been focused on individual program needs rather than on agency-wide needs.
Whatever the shortcomings of the process have been, CAAs now have computerized information infrastructures that—at least for the moment—seem to be meeting the most of the needs of the agencies that fund their various programs.
The Cost of Computerized Data Systems at CAAs. Yet for the CAAs—as well as for federal and state agencies that fund them—the development of this information infrastructure has not come without a cost. Unfortunately, there is no way within the constraints of this project that we could calculate a price tag for this infrastructure.
However, to offer just an idea of the monetary cost that has been involved, we have estimated that the replacement cost of all the computers that currently exist at all the CAAs across the state to be about $3 million. Of course, this amount does not include all the older hardware that preceded the current set of machines. And this amount does not include all the dollars spent on: printers and other peripherals; wiring; IT systems staff (or those staff who play that role de facto at CAAs where such job titles do not exist); off-the-shelf software or custom software; support and service contracts; training costs, etc.
This is not a small investment—especially for many CAAs, which have other needs, such as the need for more staff, better staff salaries, and improved physical facilities. And it is a far cry from the resources most of these organizations probably budgeted 15 or so years ago for information technology, when then the requisite tools were paper, pencils, filing cabinets, and hand calculators.
Obviously, the clock cannot be turned back—the world has changed and the new information technology is here to stay. The question is no longer: "Will tomorrow actually come?" Tomorrow is already here. Yet the new technology and the increasing demands for data that have spawned its widespread expansion within CAAs present a new series of questions. And it is these questions surrounding data collection and reporting that comprised one of the top three areas of concern expressed by Massachusetts CAAs during the assessment phase.
Given Massachusetts CAAs’ multitude of programs and funding sources—each of them with their own data reporting systems—it was beyond the scope of this project to examine each one. Instead, during the assessment phase of this project, we mainly focused on questions and issues pertaining to the area of client and services data collection and reporting as a whole, as they affect CAAs as agencies. For the most part, we did not delve into issues particular to specific programs found at CAAs except to the extent that they are part of a larger pattern.
While not every question and issue listed below was raised at every agency, nonetheless, at nearly every CAA someone brought up some combination of them. At several agencies, discussion of data-related issues consumed far more time than any other topic covered during the site visit. And when Massachusetts CAAs were asked during the site visits about issues that they would like to see their statewide Association tackle, they mentioned the area of client and services data collection and reporting issues more often than any other.
What are the specific issues raised by CAAs during the project’s assessment phase regarding client data collection and reporting? These concerns are many and can be broken down into several areas.
"Big Picture" Questions. Hardly anyone interviewed challenged the need on the part of federal and state agencies to collect data on their programs’ clients and services. However, some raised questions about the amount of data that they are required to provide, whether or not all of it is needed or useful, and how it is used. In most cases, the questions/comments were made in reference to the overall burden on the agency to provide data, and not necessarily in relationship to the requirements of a particular program. A sampling of these larger questions is as follows:
Is it really necessary to collect all this information on clients and services?
What do federal and state agencies do with all this information anyway?
How is it used to develop better public policy?
How is it used to better target services? To widen access? To get more governmental and private resources to meet our needs? To improve the quality of services? To develop a more comprehensive approach to addressing the various needs of clients who use more than one service?
Are the results worth the large investment in IT and staff time?
CAAs’ Ability to Use the Data They Collect. At many agencies, those we interviewed clearly saw the value of collecting data, and they understood how some of it either has been useful or could be useful in their own planning to improve and to better target their services. However, in some of the agencies, staff noted that in some programs, existing client and services data reporting systems established by the funding sources did not satisfy their own data needs. They described these data reporting systems as "one-way streets"— ones in which they provide information but get little or nothing in return. They pointed out that in some cases, the particular data that they were required to gather and report were not very useful to them in their planning efforts. They reported that in a few instances, they could not even retrieve data they had provided in a usable form, or at all.
Potential Adverse Impacts on Clients. Some felt that the time they were spending on data collection and reporting was beginning to encroach on time they should be spending with clients on issues besides data collection. Those that raised the issue emphasized that the situation had not reached a point where they believed services to clients were being compromised, but they felt that they were approaching that line. A few wondered aloud about how their clients are affected by the time they (the clients) have to spend providing information in this process. Of course, finding a line across which data collection needs compromise quality of services would be very difficult. Tolerance levels vary greatly among individual clients as well as staff.
Client Confidentiality. This issue emerged in some instances when discussion turned to how client data are used. Agencies that have programs in which client confidentiality is a major concern—such as HIV/AIDS services, family planning, or mental health programs—were especially cognizant of the implications of this issue. Staff at other agencies also expressed concerns. In a few cases, staff told us that they have become concerned about increasing requests from external entities to gain access to their client databases, no matter how benign or worthwhile the stated purpose of the requests appears to be.
Difficulties Surrounding the Mechanics of Data Collection and Reporting. Below is a list of various issues that were mentioned by CAA staff across the state. These pertain to overall, cross-cutting issues affecting their agencies, not those involving particular programs and their federal or state funding agencies.
Handling the burden of reporting to multiple funding sources. Many agencies complained about the growing difficulty of handling the burden of reporting to multiple funding sources (for different programs—and sometimes, multiple funding sources for the same program), which have different reporting requirements, and different software. This affects all CAAs, but it has a greater impact on larger ones: ABCD, for example, has 90 different contracts.
Lack of a standard for client demographic data. One issue that came up repeatedly is the lack of standardization of demographic data on clients requested by various funding sources. Different funding sources require different sets of data. And from funding source to funding source, the definitions of seemingly simple categories vary. For example, "family" may be defined differently by different funding sources. Racial/ethnic categories also differ in their definitions, from funding source to funding source.
Conversion issues. Staff at numerous agencies raised the following question, which is closely related to the issue described immediately above: "Can’t we find a way to more easily convert data for one funding source so it will serve the requirements of other funding sources, too?" (Of course, apart from software file format issues, this depends on developing standard data categories and definitions. Apples cannot be converted into oranges.)
Double data entry. Staff at some agencies expressed frustration at having their staff in the same program enter data twice for the same client during the same encounter—because two different funding sources for that program each require that information be submitted on their own software and in their own categories.
Problems with accurate recording. Some CAAs have staff members doing data collection and reporting who not only lack adequate computer skills—they also have difficulty in reporting data accurately, or have an insufficient understanding of their work. As a result, they cannot always identify mistakes they are making.
System Requirements Related to Data Collection. Within the past few years, some state agencies have included requirements in their contracts with CAAs and other providers regarding minimum specifications for computer technology to be used within the program covered under the contract. No one would dispute that every software package has its own minimum system requirements in terms of RAM, disk drive space, processor speed, and capabilities. However, MIS/IT staff at several CAAs questioned the minimum system specifications required by state agencies for some of their programs. These staff believe the specifications for hardware were excessive—that the required software could run on less powerful and less expensive hardware. They pointed out that such specifications make it even more difficult for CAAs to find room in their tight budgets to pay for information technology.
Massachusetts CAAs are grappling with two new and related mandates that require them to collect, compile, and report client and services data for their various programs on an agency-wide level. Massachusetts DHCD’s relatively new CSBG annual reporting requirements, along with the U.S. Department of Health and Human Services (DHHS) Office of Community Services’ (OCS) implementation of "Results-Outcomes Management Accountability" (ROMA), a new system of tracking CAA clients and reporting outcomes of services and interventions, are propelling individual CAAs to develop (or at least seriously consider developing) central, agency-wide databases that include clients from all their programs—databases that can show a count of all (unduplicated) clients served by the agency.
These federal and state requirements represent a larger, recent trend in human services—an effort to view clients in a more comprehensive way—not simply as consumers of one particular service or program, but as individuals and families that have a combination of needs that are not met merely by a single program or service. This concept has prompted an increasing number of human service providers to adopt a case management approach to service delivery. (Some CAAs have developed case management models, such as "Families First"-type programs designed to assist families in making the transition from welfare to work and self-sufficiency.)
Massachusetts CAAs are now moving to address these new requirements. Many of them are developing centralized, agency-wide client database systems. Others are moving in that direction and are examining the options. (Click here to view a sidebar showing a breakdown of the different options taken by the various CAAs.)
Yet while these requirements have propelled the majority of Massachusetts CAAs toward the development of agency-wide client tracking and reporting, at the same time they have uncovered a new set of problems. If all the existing program-based client and service data collection and reporting systems now present at CAAs are to be used as the basis for agency-wide client databases, the resulting central data system will be filled with gaps, inconsistencies, and incompatibilities.
Questions from CAA Staff. Staff at many CAAs are familiar with these obstacles. The following are significant questions that they raised:
How do CAAs seamlessly bring in data from different programs into a central database, when basic demographic information—including categories and definitions of categories—differs among each of these programs?
Incompatibilities among electronic file formats used by the different client tracking and reporting software provided by different federal and state agencies that is found within programs at CAAs still present obstacles to relatively easy importation of program data into central database systems. While it appears that technical progress is being made in this area, the problem of compiling similar data from different programs that is organized into different categories with different definitions remains.
Many CAAs do not now routinely collect client information that includes data on other family members. Several of these CAAs pointed out that if they have to begin doing that, their existing systems for many of their programs will be overwhelmed.
The CSBG annual report requires information on all programs that CSBG funds help support. Practically speaking, since CSBG funds typically support central administrations at CAAs, this requirement means that these agencies must report on virtually all their programs. However, some CAAs have programs that extend into areas for which they do not receive CSBG funding. Given this overlap in geography, which of the agencies involved reports?
In reporting for CSBG and eventually ROMA, which fiscal year should CAAs cover? Their own? The Commonwealth’s? The federal fiscal year? The fiscal year for the program involved? (Federal programs have differing fiscal years.)
For some programs, CAAs cannot utilize client data for a central client database due to restrictions placed on its use by the funding source for reasons of client confidentiality. In fact, this is the policy the Massachusetts Department of Public Health (MDPH) has adopted regarding its programs. Such MDPH programs as WIC, family planning, HIV/AIDS services, Food Stamp outreach, are found within Massachusetts CAAs, and a few, such as WIC and family planning, involve relatively large numbers of clients at several CAAs. Moreover, some federal programs have similar restrictions. Thus, at least for the foreseeable future, some CAAs will be unable to develop central client databases reflect all their clients.
For Massachusetts CAAs, client and service data collection and reporting is a topic that involves many questions and issues—some large, many small. Some of the larger issues stem from the fact that CAAs operate many services and that these services are funded and overseen by a wide array of different governmental agencies and programs—as well as some non-governmental entities. At the root of other issues lie old debates over the mission and role of CAAs as well as questions about how best to serve the needs of low-income clients in the new post-welfare reform environment of the late 1990s.
A Complex and Decentralized Human Services Infrastructure. During the last quarter-century, Massachusetts has been in the forefront of a trend among many states in contracting with nonprofit organizations, including CAAs, to provide many of the health and human services its funds. Over time, this had led to a complex, decentralized web of service delivery systems throughout the state that involves many contracted providers, large and small.
Meanwhile, the array of human services-related state agencies—and programs within these agencies—has grown. Each of these has its own mandate, priorities, and portion of the Commonwealth’s large human services budget. While many of these agencies do fall under the large umbrella entity—the Executive Office of Health and Human Services—most of them operate with considerable autonomy. And some of them have programs that enjoy relative autonomy.
The devolution of federal programs—and federal funds—to the states that has unfolded since the early 1980s has probably aided in the proliferation of state programs and their increasing compartmentalization within state government. Moreover, the increase since the 1970s in the number of groups actively advocating for increased state funding for different types of human services programs also has probably strengthened this compartmentalization.
Thus, it is not surprising that the different state entities that fund CAAs and their programs have developed their own client and services data requirements, measures, collection and reporting systems—including software. It is perfectly justifiable and reasonable that they would seek to obtain the specific client and services data that pertains to their respective missions and goals, and aids them in planning, implementation, targeting, oversight, evaluation as well as in justifying their budgets and preserving their institutional place.
Of course, while state agencies and programs have become more numerous and increasingly compartmentalized from one another, the revolution in computer technology has emerged and grown in breadth and force. The technology has made possible data collection and analysis on a scale and within a timeframe that was either impossible or impractical two or three decades ago. In this new "Information Age," state agencies and programs—like many other public and private institutions—saw the possibilities and invested in the technology. And they have tailored it to their own respective missions and needs. While the new technology may be producing wealth, it is nonetheless costly to acquire. Thus, state agencies have needed to invest significant sums in the information technology systems have developed.
It would appear that—for now, at least—the revolution in information technology probably has strengthened compartmentalization in a state human services infrastructure comprised of agencies and programs that already were becoming increasingly decentralized. The magnitude of the investment made by state agencies in developing their own respective data systems alone reinforces autonomy and a reluctance to move toward common intra-agency standards.
Countervailing Trends. At the same time, countervailing trends have emerged. As noted above, in the past decade a move has emerged in human services toward case management models and even toward greater intra-organizational collaboration. Many in the field have come to recognize that while individuals in need may be clients of a particular program or service, often they have other needs that require other services—services that are offered by a different provider. They have argued that these needs cannot be fully addressed without some form of case management for clients that considers all their needs, identifies resources, and opens doors for them.
This comprehensive approach—coupled with the recognition that the Massachusetts human services system is highly decentralized and difficult for many clients to navigate—has also prompted both state agencies and consortia of nonprofit providers to undertake ventures that involve collaboration among providers. Many of these collaborations have been launched in response to high profile issues.
Of course, the other major developments now affecting CAAs that run counter to decentralization and compartmentalization are the new OCS/DHCD annual reporting requirements and ROMA.
For both OCS and DHCD, these new requirements have obvious advantages. For CAAs, too, these new requirements have benefits, in that a by-product of centralized data collection and reporting is the fact that it helps to unify them as organizations, bolsters their overall missions and goals, and supports the case that they, in fact, are each unified entities with a purpose, not simply administrative bodies that oversee a collection of decentralized, disconnected programs. Moreover, it gives them a new means to demonstrate the vital role CAAs play in delivering services to large sectors of the population and that their services do, in fact, produce positive, measurable outcomes.
The Dilemma for CAAs. Massachusetts CAAs are caught in a dilemma between these two larger trends within the human services field and government.
On the one hand, state (and federal) agencies and programs that fund individual programs within Massachusetts CAAs have developed their own individual institutional priorities and unique data collection systems, which they in turn impose on their contracted providers. And as information technology has advanced, these different governmental entities have increased their demands for data.
On the other, Massachusetts CAAs are reacting to the trend toward more comprehensive approaches to service delivery and the case management model. Such a model has its own information requirements—ideally, a case manager should be able to easily access all the relevant information about a client and the services she/he has received or might need. (1) Meanwhile, the one funding source that all CAAs have in common—DHCD, the Massachusetts grantee for federal CSBG funds— is now placing requirements that these agencies to provide annual data on all the clients they serve in all their programs along with information that measures their progress toward desired outcomes.
At present, many of the data issues faced by Massachusetts CAAs are in large part a reflection of these two trends within their governmental funding sources—trends which seem at odds with one another. Massachusetts CAAs now carry out a difficult balancing act in attempting to comply with both. This has involved significant financial investment on the part of CAAs in computer technology to meet all the requirements of their funding sources—even when the requirements are dictated by goals of different public sector funding sources that do not seem to communicate with one another.
For many CAAs, the effort to keep pace with these requirements has consumed funds and time that they otherwise would have spent elsewhere. And despite the large financial investment that already has been made, more resources are still needed to improve CAA staff IT skills and to continue to upgrade systems.
When businesses are faced with major costs such as upgrading information technology systems, they can consider raising prices or fees. Governments can consider raising taxes or fees, float bonds, or even run up large deficits. Public utilities can seek rate increases. Even for some nonprofits, increasing fees for clients, membership, subscriptions or admissions is always an option.
CAAs can do none of the above. They are dependent upon the governmental entities with whom they contract to provide the means to cover these costs. (2) And when it is these same governmental entities that are demanding data collection and reporting that requires a high investment by CAAs in information technology, it is only reasonable to expect that they should bear the financial burden.
Of course, state agencies in Massachusetts have paid large sums to put these information systems in place. But more resources are needed if the full potential of this investment is to be realized. Both CAAs and their funding sources have a large stake in the information infrastructure that they have developed. They need to work together to ensure that the resources are available to maximize the returns on their investment, and that the system does not atrophy in the face of still more technological advances that are almost certain to occur.
For all the above reasons, the major questions and issues surrounding data collection and reporting cannot be solved by CAAs without working in close collaboration with their state and federal funding sources. Moreover, these public funding sources also must work more closely together to find ways to resolve areas where they, individually, seem to be working at cross-purposes. Given the multiplicity of government agencies and programs with which CAAs deal, it is likely that no perfect system will ever emerge. However, it should be possible to improve, simplify, streamline, and adequately fund the existing one.
Suggestions for MASSCAP: While different agencies and programs will always require data specific to their respective missions, the basic demographic data they all collect could be standardized in terms of categories and their definitions. It is unrealistic to expect such a standard any time soon that would embrace programs at both the federal and state level. However, an effort to bring the Commonwealth’s agencies and programs to adopt such a standard would seem to be a reasonable goal.
Working through MASSCAP, Massachusetts CAAs need to find a standard for client demographic data that they can agree upon. MASSCAP’s Information Technology Committee is perhaps the logical group to sort through the possibilities.
MASSCAP should reach out to other state human services provider organizations which have members that share some of the same state funding sources as CAAs and which likely have similar issues regarding data collection and reporting.
MASSCAP needs to place the issue of a common client demographic data set on the agenda with its state funding sources. Obviously, no one of these state agencies and programs is in a position to launch the intra-agency collaboration that would be necessary for a standard to come about, but dialogue about this issue could help pave the way for such collaboration to take place.
MASSCAP should engage in a regular, ongoing dialogue with DHCD about the role it can play in such an effort. DHCD has a particular stake in seeing the development of agency-wide client database systems at CAAs, and adoption of a standard demographic data set would aid this effort. This dialogue should also focus on resolving many of the unanswered questions raised by CAAs during the assessment phase of this project (and noted above in this section) relating to CSBG annual report data collection.
MASSCAP should identify appropriate officials within the Executive Branch—such as EOHHS or A&F—as well as other officials and begin a dialogue around the issue. MASSCAP should also raise the issue with key legislators, especially those with oversight over health, human services, and other agencies with which CAAs contract. If intra-agency collaboration is to take place, it will need to be initiated by an entity within state government that has the power to bring different state programs and agencies together. At the legislative level, the issue might be a topic for the Committee on Human Services and Elder Affairs to examine.
Individual CAAs that do not have a standard client intake form for purposes of CSBG annual reporting that includes the same demographic categories (and definitions) and that is used consistently in all programs throughout their respective agencies should consider developing one. (A sample from Citizens for Citizens appears in the appendices of this report.) At minimum, CAAs should develop consistency in basic data collection throughout their respective organizations. This will be useful for annual CSBG reporting and ultimately for ROMA.
CAAs that have not developed central client databases should at least consider doing so and investigate the options available.