Inside IES Research

Since 2014, IES has funded the Center for the Analysis of Postsecondary Readiness (CAPR) to answer questions about the rapidly evolving landscape of developmental education at community colleges and open-access four-year institutions. CAPR is providing new insights into how colleges are reforming developmental education and how their reforms are impacting student outcomes through three major studies:

• A survey and interviews about developmental education practices and reform initiatives
• An evaluation of the use of multiple measures for assessing college readiness
• An evaluation of math pathways.

Preliminary results from these studies indicate that some reforms help more students finish their developmental requirements and go on to do well in college-level math and English.

National Study of Developmental Education Policies and Practices

CAPR has documented widespread reform in developmental education at two- and four-year colleges through a national survey and interviews on developmental education practices and reforms. Early results from the survey show that colleges are moving away from relying solely on standardized tests for placing students into developmental courses. Colleges are also using new approaches to delivering developmental education including shortening developmental sequences by compressing or combining courses, using technology to deliver self-paced instruction, and placing developmental students into college-level courses with extra supports, often called corequisite remediation.

Developmental Math Instructional Methods in Public Two-Year Colleges (Percentages of Colleges Implementing Specific Reform Strategies)

Notes: Percentages among two-year public colleges that reported offering developmental courses. Colleges were counted as using an instructional method if they used it in at least two course sections. Categories are not mutually exclusive.

Evaluation of Developmental Math Pathways and Student Outcomes

CAPR has teamed up with the Charles A. Dana Center at the University of Texas at Austin to evaluate the Dana Center Mathematics Pathways (DCMP) curriculum at four community colleges in Texas. The math pathways model tailors math courses to particular majors, with a statistics pathway for social science majors, a quantitative reasoning pathway for humanities majors, and an algebra-to-calculus pathway for STEM majors. DCMP originally compressed developmental math into one semester, though now the Dana Center is recommending corequisite models. Instructors seek to engage students by delving deeply into math concepts, focusing on real-world problems, and having students work together to develop solutions.

Interim results show that larger percentages of students assigned to DCMP (versus the traditional developmental sequence) enrolled in and passed developmental math. More of the DCMP students also took and passed college-level math, fulfilling an important graduation requirement. After three semesters, 25 percent of program group students passed a college-level math course, compared with 17 percent of students assigned to traditional remediation.

Evaluation of Alternative Placement Systems and Student Outcomes (aka Multiple Measures)

CAPR is also studying the impact of using a combination of measures—such as high school GPA, years out of high school, and placement test scores—to predict whether students belong in developmental or college-level courses. Early results from the multiple measures study show that, in English and to a lesser extent in math, the multiple measures algorithms placed more students into college-level courses, and more students passed those courses (compared to students placed with a single test score).

College-Level English Course Placement, Enrollment, and Completion in CAPR’s Multiple Measures Study (Percentages Compared Across Placement Conditions)

College-Level Math Course Placement and Completion in CAPR’s Multiple Measures Study

Looking Ahead to the Future of Developmental Education

These early results from CAPR’s evaluations of multiple measures and math pathways suggest that those reforms are likely to be important pieces of future developmental education systems. CAPR will release final results from its three studies in 2019 and 2020.

Guest blog by Nikki Edgecombe and Alexander Mayer

Nikki Edgecombe is the principal investigator of the Center for the Analysis of Postsecondary Readiness, an IES-funded center led by the Community College Research Center (CCRC) and MDRC, and a senior research scientist at CCRC. Alexander Mayer is the co-principal investigator of CAPR and deputy director of postsecondary education at MDRC.

As part of our series recognizing Career and Technical Education (CTE) month, we interviewed Mary Visher, Senior Associate at MDRC, about her recently funded study, Teaching and Learning 21st Century Skills in Community Colleges: A Study of the New World of Work Program (NWoW). This project is developing and testing an innovative program aimed at teaching 21^st century skills to community college students in CTE courses. Researchers, policymakers, and practitioners are working closely together to improve upon NWoW, which is already in use in several community colleges in California.

The Center for Analysis of Postsecondary Education and Employment (CAPSEE) is an IES-funded Research and Development Center that seeks to advance knowledge regarding the link between postsecondary education and the labor market. CAPSEE was funded through a 2011 grant from the National Center for Education Research (NCER) and is in the process of completing its work. CAPSEE will hold a final conference to discuss its findings on April 6 & 7 in Washington, DC.

Recently, Tom Bailey (pictured), Director of the Community College Research Center, Columbia University, Teachers College, and the Principal Investigator for CAPSEE, answered questions from James Benson, the NCER Program Officer for the R & D center.

Can you describe some of the original goals of CAPSEE?

We were especially interested in the economic benefits of a college education for community college students, including those who complete awards (Associate’s degrees or certificates) and those who do not, as well as those who transfer to four-year colleges. We were also interested in differences in earnings by field of study. When we started CAPSEE in 2012 there were a lot of studies that used survey datasets to look, in general, at the returns to completing a Bachelor’s degree. The CAPSEE approach was to use large-scale statewide databases and follow college students over time, to look in detail at their earnings before, during, and after college.

In addition, CAPSEE researchers sought to examine two key policy issues. One was how financial aid and working while enrolled affect students’ performance in college and their labor market outcomes. The other was whether for-profit colleges help students get better jobs.

You have synthesized findings from analyses in six states. What are your main findings?

We found that, in general, Associate’s degrees have good returns in the labor market; they’re a good investment for the individual and for society. However, there is quite a bit of variation in returns by program. For students in Associate degree programs primarily designed to prepare them for transfer to a four-year college, if they don’t transfer, their degrees will not be worth very much. But when they complete vocational degrees, especially in health-related fields, the earnings gains are usually strong and persistent (and robust to how we estimated them). Also, we did a lot of research on certificates, credentials that many see as the best fit for students on the margin of going to college. We found benefits to students who completed certificates, again especially in fields that directly relate to an occupation or industry. And finally, we examined outcomes for students who enrolled and took courses without attaining a degree or certificate. We found that their after-college earnings increased in proportion to the number of credits they earned.

"The fundamental policy implication is that college is a good investment."

What do you see as the key policy implications of these findings?

The fundamental policy implication is that college is a good investment. This merits emphasis because there are repeated critiques of college in terms of how much it costs and how much debt students accumulate. That said policymakers do need to think about the value of each postsecondary program. Even within the same institution, programs have very different outcomes. Yet on average, attending college for longer and attaining more credits has beneficial effects. Policymakers should see this evidence as supporting public and private investments in college.

What did you discover about the relationships between financial aid, college outcomes, and labor market outcomes?

In an era of tight public resources, the effectiveness of financial aid policy is a crucial issue. Financial aid does help students persist in college, but one way to promote greater effectiveness is through academic performance standards for students receiving federal financial aid. These have existed in the federal need-based aid programs for nearly 40 years, in the form of Satisfactory Academic Progress (SAP) requirements. These have not received much attention. Our research on SAP suggests such policies have heterogeneous effects on students in the short term: they increase the likelihood that some students will drop out, but appear to motivate higher grades for students who remain enrolled. After three years, however, the negative effects dominate. Though it has little benefit for students in the long term, SAP policy appears to increase the efficiency of aid expenditures because it discourages students who have lower-than-average course completion rates from persisting. But the policy also appears to exacerbate inequality in higher education by pushing out low-performing, low-income students faster than their equally low-performing, higher-income peers.

Many students work while in college.  Does this seem to help or hurt students in the long run?

Our research found that for Federal Work-Study (FWS) participants who would have worked even in the absence of the program, FWS reduces hours worked and improves academic outcomes but has little effect on post-college employment outcomes. For students who would not have worked, the effects are reversed: the program has little effect on graduation, but a positive effect on post-college employment.  Results are more positive for participants at public institutions, who tend to be lower income than participants at private institutions. Our findings suggest that better targeting to low-income and lower-scoring students could improve FWS outcomes. This is consistent with much of the CAPSEE research—you need more detail and specificity to really understand the relationship between education and employment and earnings.

What did you learn about credentials from for-profit institutions?

Our findings on students at for-profit colleges were quite pessimistic. Although enrollment in for-profit colleges grew significantly after 2000, the sector has been declining during the last two years, as evidence on inferior outcomes – particularly with regard to student debt – emerged. In general, our researchers found that for-profit students have worse labor market outcomes than comparable community college students although in some cases the difference is not statistically significant. Our evidence suggests that these colleges need to be monitored to ensure they are delivering a high-quality, efficient education.

You are holding the final CAPSEE conference in April. What do you hope people will get out of it?

At the conference, we will focus on several important and controversial policy questions related to higher education: 

• Have changes in tuition and the labor market created conditions in which college is not worth it for some students, contributing to an unsupportable increase in student debt? 
• Has higher education contributed to inequality rather than promoting economic mobility? 
• Is continued public funding of college a worthwhile investment? 
• Should public funding be used only for some programs of study?  
• What are the arguments for and against making community college free?
• Can changes in the operations and functioning of colleges change the return on investment from a college education for both the individual and society?
• How important should information on earnings outcomes be for accreditation decisions and/or for eligibility of students to receive financial aid? 

At the conference participants will have the opportunity to discuss and learn about these issues drawing on five years of CAPSEE research as well as input from other experts.

By Meredith Larson, NCER Program Officer                                                                                     

Although more students with disabilities are pursuing postsec ondary education, completion rates for this group of students have not changed very much in recent years. In a two-year study funded through an IES grant, Lynn Newman, of SRI International, and Joseph Madaus, director of the Center on Postsecondary Education and Disability at the University of Connecticut, have examined the impact that supports and accommodations have had on the postsecondary success of students with disabilities.

             

 

At the heart of their study is the National Longitudinal Transition Study-2 , the largest and richest data set available to address the postsecondary experiences and outcomes of youth with disabilities. It is the only dataset that can address those topics for postsecondary students with disabilities nationally, independent of students’ decisions to disclose a disability to their postsecondary school.

Below are excerpts from an email interview with the researchers. 

What motivated your study, and what questions are you grappling with?

Although postsecondary enrollment rates for students with disabilities have increased dramatically for youth in all disability categories over the past two decades, postsecondary completion rates for students with disabilities have remained stagnant over time. These students continue to be less likely to graduate from postsecondary school than their general population peers.

This led us to ask

• What is the link between receipt of postsecondary supports and accommodations – both those available because of a disability and those available to the general student body – and postsecondary persistence and completion for students with disabilities?
• What factors are associated with requesting/receiving postsecondary supports and accommodations?

What are your major findings?

First and foremost, students with disabilities who received supports, particularly supports available to the full student body (such as tutoring and access to writing and study centers), are more likely to persist in and complete their postsecondary programs. This finding applies to students with disabilities enrolled at both 2-year and 4-year colleges.

However, we didn’t find a significant relationship between receipt of disability-specific supports and accommodations (such as test accommodations, readers, interpreters) and postsecondary persistence or completion for the full population of students with disabilities. We found that the link between supports/accommodations and outcomes differs by disability category. For example, students who were deaf or hard of hearing and received disability-specific accommodations and supports were more likely to persist in or complete postsecondary education than were those who had not received these types of help.  

Does your research suggest why some students seek out or use supports more than others?

Fewer than half of those with disabilities in postsecondary institutions accessed the types of supports available to the general student body, and less than one-quarter received disability-specific help during postsecondary school. Students who received transition planning education in high school and those whose transition plans specified needed postsecondary supports and accommodations were significantly more likely to access both generally-available and disability-specific supports in postsecondary school, particularly at 2-year institutions.

If you could tell each of your target audiences what your research means for them in practical terms, what would you say?

Students and families: By accessing supports and help at postsecondary institutions,  you increase your odds for postsecondary success. If you are uncomfortable sharing information about your disability, which is required to receive disability-specific supports, you should, at least, access the types of supports available to the general student body, such as tutoring and writing centers.

High school staff: Help students avail themselves of supports at the postsecondary level through transition planning. Transition planning education and transition plans that specify postsecondary accommodation needs significantly affect whether students seek postsecondary supports. Clearly, the transition education and planning you can do matters. However, as many as one third to one half of high school students with disabilities do not receive such transition planning services.

Postsecondary staff: Keep in mind that only 35% of students with disabilities who received services in high school disclosed their disability to their postsecondary institutions, so you probably have more students with disabilities on your campus than you may be aware of. Because receipt of postsecondary supports (especially general supports available to all students) are particularly beneficial to students with disabilities, we encourage active and broad outreach about these supports to the entire student body, rather than focusing on just the few students who have chosen to disclose their disability.

In addition, we encourage professional development for postsecondary staff, particularly those involved in providing generally available supports, to help them better recognize and support students with disabilities.

Researchers: Consider the representativeness of your samples of postsecondary students with disabilities. If respondents are identified through self-disclosure of a disability, your sample probably has a large amount of underreported students with disabilities overall. Your sample is also likely to be biased, in that students with more visible disabilities are much more likely to disclose their disability than are those in the higher incidence disability categories, such as learning disabilities.

What might some next research steps be?

Given our findings, we believe there are many opportunities for research related to postsecondary education for students with disabilities.  For example, researchers could study questions about the characteristics, content, extent, and timing of effective postsecondary supports and accommodations. The field would also benefit from additional knowledge about effective high school transition planning education and answers to questions about the characteristics and structures of high schools and postsecondary schools that offer effective supports and accommodations and transition planning education. 

Questions? Comments? Please send them to IESResearch@ed.gov.

By Katina Stapleton and James Benson, NCER Program Officers

A core problem for research on minority-serving institutions (MSIs) is that they have been defined inconsistently. Through the IES-funded Center for Analysis of Postsecondary Education and Employment (CAPSEE) at Teachers College, Columbia University, researcher Valerie Lundy-Wagner is leading two research projects that aim to provide the definitional and contextual information necessary for carrying out more comprehensive and rigorous research on MSIs and the ethnic/racial and low-income students they disproportionately serve.

We spoke with Valerie about her motivation for studying MSIs and the challenges that face MSI researchers.

How did you become interested in studying minority-serving institutions (MSIs)?