EDInternet Customer Survey Results -- September 2003 Update

DATE:	October 22, 2003
TO:	Arthur Graham Director, OCIO/Information Management
THROUGH:	Sally Budd Director, Development Services Group
FROM:	Keith M. Stubbs Information Technology Specialist, Development Services Group
SUBJECT:	Internet Customer Survey Results - September 2003 Update

More than 4,000 customers have responded to the Department's Internet customer survey since November 1996. The purpose of this memorandum is to update the October 30, 2002 report on the first 3,782 responses, which were received from November 1996 through mid-October 2002, and examine the 249 responses received in the 10½ months between then and the launch of the redesigned www.ed.gov website on September 7, 2003.

Highlights

The survey responses received in the last 10½ months indicate that:

• Our most frequent users are students (26%), parents (24%), teachers and professors (12%), education administrators and managers (8%), and researchers and analysts (6%). To a lesser extent, our users include librarians (2%), writers and reporters (1%), policy makers and legislators (<1%), and miscellaneous others (19%) including counselors and consultants. Most student respondents are at the college level rather than the elementary and secondary level. Most administrators/managers and teachers are at the elementary and secondary level. Most researchers are affiliated with colleges, associations, or for-profit or non-profit organizations.

• Our users' organizational affiliation is private individuals (29%), colleges and universities (20%), elementary and secondary education (21%), junior and community colleges (6%), associations and non-profit organizations (4%), for-profit organizations and businesses (4%), federal government (3%), libraries (3%), state government (2%), White House or Congress (1%), local government (<1%), and miscellaneous others (7%).

• The majority of respondents visit ED's Web site at least once a month; 45% visit it at least once a week. This year's responses show a modest 3% shift toward more frequent visits. Administrators/managers and researchers tend to visit ED's Web site more frequently than the average respondent. Students tend to visit less frequently, which is consistent with the notion that most students visit solely for financial aid.

• Overall satisfaction dropped from 3.56 (on a scale of 1-5) in last year's responses to 3.51 in this year's responses. Administrators/managers and teachers were the most satisfied customers. Students were the least satisfied customers.

  Satisfaction was highest in the five presentation (3.60) and responsiveness (3.51) categories and slightly lower in the information content (3.44) and organization (3.37) categories. The highest-rated individual categories were web screen responsiveness (3.86), search responsiveness (3.75), clarity of writing (3.70), clarity of graphics (3.67), clarity of tables/charts (3.59), and accuracy (3.57). The lowest-rated categories were ease of finding information of interest (3.23), relevance to needs (3.29), menus and categories (3.37), comprehensiveness, (3.39), usefulness of links (3.40), and search tools (3.41).

  Comments indicate that the small drop in satisfaction is attributable to (a) raised customer expectations for web sites in general, (b) mixed response to the October 2001 redesign, and (c) a shift in customer needs from the relatively mature information offerings of the ED web site to the more complex student aid services and other e-commerce offerings that the new e-Government initiatives are starting to address.

• Respondents consider most kinds of information ED provides to be useful or very useful (2.36 on a scale of 1 to 3). The overall usefulness rating dropped slightly this year from 2.31 to 2.27 this year. Ratings dropped for 13 of the 19 categories. Ratings rose most for legislation and regulations and descriptions of ED-funded projects. Ratings dropped most for statistics, research findings, lesson plans and teacher guides, and research reports.

• When asked how ED's Internet services should be improved, respondents suggested:
  

  • putting all ED information online promptly and keeping it up-to-date;
  • presenting information in layman's terms with simple explanations of complicated rules and procedures;
  • providing a tutorial or quick tour for first-time visitors to help them navigate the site;
  • providing an orientation to the Department, including its mission, scope, organization, functions, history, etc.;
  • providing complete contact information for all Department officials, programs, services, activities, and investigative/enforcement functions;
  • improving responsiveness to customer inquiries;
  • improving ease of use for student financial aid services and making it easier to find specific student aid services;
  • providing comparative rating and ranking information about schools, districts, and states;
  • using the web to support the full grant lifecycle, including an always-current forecast of funding opportunities, comprehensive, up-to-date information about each grant program in a consistent format, and information to help prospective applicants;
  • offering more information and resources of direct utility to parents and teachers; and
  • providing a more comprehensive gateway to education information and services elsewhere

• See Attachment B for selected "Rants & Raves" representing the most positive and negative overall comments in the 249 responses received in the last 10½ months.

Purpose of the Survey

Web usage logs provide much useful data - the number of times each file is accessed, when, from which Internet address, and using which Web browser. From that data we can derive the kind of information included in the Web Services Group's monthly reports - the growth in customer traffic over time, the most frequently used files, and the most widely used Web browsers among our customers.

The Internet Customer Survey (Attachment C) was designed to provide insight into some important customer characteristics that we cannot glean from the usage logs:

• What kinds of people and organizations use our information?
• How often do they visit ED's Web sites? Do they subscribe to the EDInfo email list?
• How satisfied are they with the information we provide - its usefulness, timeliness, accuracy, comprehensiveness, and presentation?
• How satisfied are they with the way the information is organized and with the menus, links, and search tools we provide to help them find information of interest?
• What kinds of information are most valuable to them?
• How do they prefer to use various types of information, and what formats are most useful? What is their technical capacity to receive and use the information?
• What kinds of new services would be most valuable to them in the future?

Conducting the Survey

The survey, which the Internet team designed in consultation with the National Center for Education Statistics (NCES) and the Office of the Under Secretary (OUS), was approved by the Office of Management and Budget (OMB) and made available on the ED Web site in November 1996. Since then OMB has re-approved the survey twice. An updated analysis of the responses is issued every year.

What the Responses Tell Us

The survey cannot be considered a random sample, since respondents were visitors to the ED Web site who took the time to respond.

This memorandum focuses on the responses received in the last year. However, cumulative tabulations of all 4,031 responses are included on the assumption that they help provide a balanced view of our customer base over time.

The response rate fell from 6.4 per week last year to 5.4 per week this year.

• The number of respondents in the policy maker, librarian, and writer categories in the response groups the last two years was very small (fewer than eight per year). The small number of respondents tends to make the categories appear as outliers, e.g., extremely high or low satisfaction ratings. Therefore, those groups are excluded from most of the tables and statements below.

• The cumulative responses to Questions #7 through 10, which ask customers to rate their satisfaction in eighteen categories covering all aspects of the ED Web site, show that overall satisfaction over the duration of the survey has been high - 3.75 on a scale of 1 to 5, where 5=very satisfied, 4=satisfied, 3=neither satisfied nor dissatisfied, 2=dissatisfied, and 1=very dissatisfied.

• Overall satisfaction in this year's responses was 3.51, down slightly from last year (3.56). Administrators/managers and teachers were the most satisfied customers. Students were the least satisfied customers.

• Across all eighteen categories, the general distribution of this year's responses was 59% satisfied or better, 22% neutral, and 19% dissatisfied or worse - a 1% shift toward dissatisfaction from last year.

• Surprisingly, satisfaction for the responsiveness categories appears to vary inversely with the users' connection speed. Users with direct Internet connections are less satisfied than users with modems, and users with fast modems are less satisfied than users with slower modems. It appears that users with direct connections may have higher expectations of web sites than do modem users.

• No responses were excluded from the results, even those that voice bitter opposition to the Department's existence and gave the lowest available rating in all categories. See Attachment B for selected "Rants & Raves" representing the most positive and negative overall comments in the 249 responses received in the last 10½ months.

• Based on this year's responses, our most frequent users are students (26%), parents (24%), teachers and professors (12%), education administrators and managers (8%), and researchers and analysts (6%). To a lesser extent, our users include librarians (2%), writers and reporters (1%), policy makers and legislators (<1%), and miscellaneous others (19%) including counselors and consultants.

• This year's responses include a lower percentage of education administrators/managers (-4%), researcher/analysts (-4%), and students (-1%) and a higher percentage of parents (+6%) and teachers (+4%) than last year's responses.

• Most student respondents are at the college level rather than the elementary and secondary level. Most administrators/managers and teachers are at the elementary and secondary level. Most researchers are affiliated with colleges, associations, or for-profit or non-profit organizations.

• Based on this year's responses, our users' organizational affiliation is private individuals (29%), colleges and universities (20%), elementary and secondary education (21%), junior and community colleges (6%), associations and non-profit organizations (4%), for-profit organizations and businesses (4%), federal government (3%), libraries (3%), state government (2%), White House or Congress (1%), local government (<1%), and miscellaneous others (7%).

• The majority of respondents visit ED's Web site at least once a month; 45% visit it at least once a week.

• This year's responses show a modest 3% shift toward more frequent visits.

• This year's responses show that administrators/managers and researchers tend to visit ED's Web site more frequently than the average respondent; students tend to visit less frequently. That is consistent with the notion that most students visit solely for financial aid.

• More than half of our customers have direct Internet connections; the rest use modem connections.

• Current and former subscribers to the EDInfo listserv are much more likely to be frequent users of ED's Web site than respondents who never subscribed to EDInfo.

Category	Avg. Rating* (Rank)			10/16/2001 through 10/15/2002 (last year)	10/16/2002 through 9/7/2003 (this year)
	Cum.	Last Year	This Year
Relevance to your needs	3.65 (13)	3.43 (14)	3.29 (17)	Rated highest by Teachers (4.08) Researchers (3.84) Managers (3.84) Rated lowest by Parents (3.22)	Rated highest by Managers (4.11) Rated lowest by Students (3.18) Parents (3.19)
Timeliness	3.85 (5)	3.62 (9)	3.54 (8)	Rated highest by Teachers (4.17) Researchers (4.03) Managers (3.84) Rated lowest by Students (3.48)	Rated highest by Managers (3.94) Teachers (3.88) Rated lowest by Parents (3.35) Students (3.42)
Accuracy	3.91 (2)	3.63 (8)	3.57 (6)	Rated highest by Teachers (4.13) Researchers (3.86) Managers (3.81) Rated lowest by Parents (3.42)	Rated highest by Managers (4.11) Rated lowest by Parents (3.45) Students (3.45)
Completeness/ Comprehensiveness	3.64 (14)	3.43 (14)	3.39 (15)	Rated highest by Teachers (3.96) Managers (3.72) Rated lowest by Parents (3.29)	Rated highest by Managers (4.06) Teachers (4.00) Rated lowest by Parents (3.21) Students (3.28)
Overall usefulness	3.69 (10)	3.45 (12)	3.42 (12)	Rated highest by Teachers (4.21) Managers (3.78) Researchers (3.71) Rated lowest by	Rated highest by Managers (4.11) Researchers (3.70) Rated lowest by Parents (3.34)
Total: all 5 information categories	3.75	3.51	3.44	Rated highest by Teachers (4.11) Researchers (3.83) Managers (3.80) Rated lowest by Parents (3.38)	Rated highest by Managers (4.07) Teachers (3.76) Rated lowest by Parents (3.31) Students (3.34)

*  5=very satisfied, 4=satisfied, 3=neither satisfied nor dissatisfied, 2=dissatisfied, and 1=very dissatisfied. Parenthesized number indicates category's rank among 18 satisfaction categories.

• Satisfaction dropped slightly in all five categories in this year's responses. Comments indicate that the drop is attributable to (a) raised customer expectations for web sites in general, (b) mixed response to the October 2001 redesign, and (c) a shift in customer needs from the relatively mature information offerings of the ED web site to the more complex student aid services and other e-commerce offerings that the new e-Government initiatives are starting to address.

• The percentage of dissatisfied customers in this year's responses was highest for relevance (27%), comprehensiveness (25%), and overall usefulness (24%); fewer customers were dissatisfied with timeliness (17%) and accuracy (17%). Some of the dissatisfaction appears to stem from confusion about the federal role in education.

• This year, administrators/managers and teachers tend to be most satisfied with the information that ED currently provides. Parents are least satisfied.

• A representative sample of suggestions for improvement:

• General comments
  
  • Does a good job, just keep information updated
  • Generally speaking, I find it very hard to make much sense of much of the information presented. It's almost as if the website is for educational professionals only -- the "admin-speak" language needs to be toned down.
  • It to distracting, too many vivided areas on the screen. too "busy" complicated to navigate and find relevant info. looks like a web engine home page with lots of ads and links. this page needs to be re-done and simplified.
  • The website is very comfortable on my eyes
  • There wasn't much info.
  • The site is very overwhelming
  • Some of us don't have the time to browse a site for too long, if I don't see the information, or at lease a lead, I seek within 5 minutes, I have to move on or inquire e-mail.
• General information
  
  • I can not find the definition of education on the web site
  • I was looking for information on the history of the Department of Education and was not able to find it - i.e. when it was founded, who was the President, etc.
  • Can't find specific links to obtaining educational history, and credits etc. Is there such a place other than obtaining transcripts from the last school attended?
• Contacts, alerts & interactivity
  
  • How do I email Ron Paige???? Where's the email address???
  • It would be helpful to have e-mail addresses for your employees. We have filed a complaint for our son with your office and it's hard for me to make phone calls during the day. It would be so much easier to check the progress of the investigation if I were able to e-mail the person in charge of his case.
  • Need to make it easier to contact specific subject areas for questions.
  • I find it hard to find specific information. For example I'm have been trying to find information regarding Perkins assignments beyond the regulation handbook. The contact department listed with the assignment guidelines knows nothing and why they list them as a source is beyond me. I cannot locate anything on the web site nor can I figure out what dept I should contact. It may simple be there nothing available outside the written regs but a knowledgeable contact person would be greatly appreciated.
  • It would be nice if the "Electronic Phonebook" could be searched using a catagory such as "student loans" instead of just searching for individuals.
• State & local information
  
  • I'm looking for the site where I can find the educational ratings for each state as to the best state to the worse in education.
  • Exemplary as well as deficient schools for my area are of particular interest as a parent and community involved citizen.
  • I would like to find info on how my school district is doing and was not able to find it.
  • it needs more info to help any one. If there looking for new schools. and how many children go there. who runs it and so one.
  • I tried finding something fairly simple: a listing of US Public High Schools -- and could find nothing!
• Helping teachers & students
  
  • Grade specific information would be handy since teachers only get a small bit of the information but are responsible for carrying through legislation without seeing it.
  • I clicked on this sight hoping to find some help for my son who attends a school in Rapid City and is having trouble in reading at his grade level. His I.Q is 113 and in 5th grade but his reading level is 2nd grade and his math skills are 3rd to 4th grade. I'm leaving this sight with no direction. I'll keep searching.
  • Information content I seek is how to verify that a vocational school I wish to attend is for real. I did not find any place to look for this.

Category	Avg. Rating* (Rank)			10/16/2001 through 10/15/2002 (last year)	10/16/2002 through 9/7/2003 (this year)
	Cum.	Last Year	This Year
Clarity of the writing (readability, ease of interpretation)	4.10 (1)	3.86 (1)	3.70 (3)	Rated highest by Teachers (4.25) Managers (3.91) Rated lowest by	Rated highest by Teachers (4.30) Rated lowest by Parents (3.53)
Layout of the material	3.89 (4)	3.64 (6)	3.49 (10)	Rated highest by Teachers (4.08) Managers (3.81) Rated lowest by	Rated highest by Managers (4.29) Rated lowest by Students (3.32)
Clarity of the tables and charts	3.91 (2)	3.73 (3)	3.59 (5)	Rated highest by Teachers (4.13) Researchers (4.00) Rated lowest by Parents (3.52)	Rated highest by Managers (4.29) Rated lowest by Researchers (3.38) Students (3.42)
Amount of graphics (too few, too many)	3.72 (8)	3.64 (6)	3.57 (6)	Rated highest by Teachers (4.00) Rated lowest by Parents (3.49)	Rated highest by Managers (4.06) Rated lowest by Researchers (3.38)
Clarity of the graphics	3.81 (6)	3.71 (4)	3.67 (4)	Rated highest by Teachers (4.13) Rated lowest by Parents (3.48)	Rated highest by Managers (4.12) Rated lowest by Parents (3.50)
Total: all 5 presentation categories	3.89	3.71	3.60	Rated highest by Teachers (4.12) Researchers (3.79) Rated lowest by Parents (3.58)	Rated highest by Managers (4.18) Rated lowest by Researchers (3.48) Parents (3.48)

*  5=very satisfied, 4=satisfied, 3=neither satisfied nor dissatisfied, 2=dissatisfied, and 1=very dissatisfied. Parenthesized number indicates category's rank among 18 satisfaction categories.

• Although ratings dropped slightly in all five presentation categories in this year's responses, respondents remain more satisfied with the presentation of the information than with the other satisfaction categories.

• This year, administrators/managers and teachers tend to be most satisfied with the presentation of the information that ED currently provides. Parents and researchers are least satisfied.

Category	Avg. Rating* (Rank)			10/16/2001 through 10/15/2002 (last year)	10/16/2002 through 9/7/2003 (this year)
	Cum.	Last Year	This Year
Ease of finding information of interest	3.58 (17)	3.29 (18)	3.23 (18)	Rated highest by Teachers (4.04) Parents (3.40) Rated lowest by Researchers (3.03)	Rated highest by Managers (3.83) Rated lowest by Researchers (2.63) Students (3.18)
Ease of finding new material	3.61 (15)	3.37 (17)	3.44 (11)	Rated highest by Teachers (4.00) Rated lowest by Researchers (3.17)	Rated highest by Managers (4.00) Rated lowest by Students (3.23)
Menus and categories (clarity, ease of use)	3.67 (11)	3.45 (12)	3.37 (16)	Rated highest by Teachers (3.96) Managers (3.58) Rated lowest by Researchers (3.17)	Rated highest by Managers (4.00) Rated lowest by Students (3.09)
Links (relevance, usefulness)	3.70 (9)	3.47 (11)	3.40 (14)	Rated highest by Teachers (4.00) Managers (3.73) Rated lowest by Researchers (3.37)	Rated highest by Managers (3.94) Rated lowest by Students (3.24) Parents (3.29)
Search tools	3.57 (18)	3.43 (14)	3.41 (13)	Rated highest by Teachers (3.87) Managers (3.57) Rated lowest by	Rated highest by Managers (3.94) Rated lowest by Researchers (3.25) Students (3.26)
Total: all 5 organization categories	3.63	3.40	3.37	Rated highest by Teachers (3.97) Managers (3.51) Rated lowest by Researchers (3.21)	Rated highest by Managers (3.94) Teachers (3.59) Rated lowest by Students (3.20)

*  5=very satisfied, 4=satisfied, 3=neither satisfied nor dissatisfied, 2=dissatisfied, and 1=very dissatisfied. Parenthesized number indicates category's rank among 18 satisfaction categories.

Category	Avg. Rating* (Rank)			10/16/2001 through 10/15/2002 (last year)	10/16/2002 through 9/7/2003 (this year)
	Cum.	Last Year	This Year
Web/gopher screens (quick/sluggish)	3.76 (7)	3.81 (2)	3.86 (1)	Rated highest by Managers (4.20) Teachers (4.17) Researchers (4.07) Rated lowest by Parents (3.57)	Rated highest by Managers (4.13) Rated lowest by Parents (3.63)
Searches (quick/sluggish)	3.67 (11)	3.69 (5)	3.75 (2)	Rated highest by Researchers (4.15) Teachers (4.09) Managers (3.92) Rated lowest by Parents (3.52)	Rated highest by Managers (4.27) Researchers (3.88) Rated lowest by Parents (3.62)
Webmaster@inet.ed.gov (email inquiries: speed and quality of reply)	3.61 (15)	3.53 (10)	3.50 (9)	Rated highest by Teachers (3.95) Managers (3.92) Researchers (3.73) Rated lowest by Students (3.39)	Rated highest by Managers (3.78) Teachers (3.69) Rated lowest by Researchers (3.13) Parents (3.32)
Total: all 3 responsiveness categories	3.69	3.69	3.72	Rated highest by Teachers (4.07) Managers (4.05) Researchers (4.03) Rated lowest by Parents (3.55)	Rated highest by Managers (4.10) Rated lowest by Parents (3.53) Researchers (3.58)
Total: all 18 satisfaction categories	3.75	3.56	3.51	Rated highest by Teachers (4.07) Managers (3.75) Researchers (3.67) Rated lowest by	Rated highest by Managers (4.07) Teachers (3.80) Rated lowest by Students (3.40) Parents (3.42)

*  5=very satisfied, 4=satisfied, 3=neither satisfied nor dissatisfied, 2=dissatisfied, and 1=very dissatisfied. Parenthesized number indicates category's rank among 18 satisfaction categories.

• Ratings rose this year for ease of finding new material, speed of searches, and speed of web screens, while dropping for the other fifteen satisfaction categories. Ratings dropped most for relevance, layout of the material, and clarity of the tables, charts, and writing.
• This year, administrators/managers and teachers tend to be most satisfied with the organization and finding aids on the ED web site. Students are least satisfied.
• Administrators/managers and teachers tend to be most satisfied with responsiveness. Parents and researchers are least satisfied.
• The ratings and comments in the organization categories reflect the difficulty of organizing a large body of information into a scheme that works well for such a broad range of customers. Some representative comments are:
• General organization
  
  • I really liked the way the material was organized. In particular, the highlighted content was helpful.
  • The website is a bit crowded and not well organized. Sometimes it is difficult to locate the information that I am looking for.
  • it needs to be spred out better
  • Information, directions not user friendly for people who are not familiar with this site.
  • Too much crap to deal with on main page. You should make it easier to use and find what is needed and the go to a specific page that deals with one thing only. Need better directions on how to use the site and not automatically assume the user knows the government terms you are currently using or the names of the particular program.
  • Too difficult for layperson to find information to our specific needs for our family situation - simplify choices and/or at the least put a description that we can understand or it takes for ever to find anything.
• Student aid
  
  • I log on to the website to find out about my student loan,it ususally takes about 1/2 to navigate to finally 'accidentaly' find it. It needs to STAND OUT, be easier to find, or include a link, if I'm not using the right page.
  • I am trying to find out about my status for this fall term of 2003-2004, i am having trouble finding out where to go to find the information. i feel i am just going in circles
  • Have student loan accounts accessible on this site. I am well educated, but can not find the time to get through all the junk on this website to find out about my loans.
  • Why can't I easily find information on repaying my loan??? After following various links and then finally getting to the "repayment" page, the only information listed was for defaulted loans, or consolidating loans, or finding out about loans you might apply for. Isn't the department interested in making it easy for those of us trying to repay our loans get information on it??? I was looking for the 800 number to call and find out about my own loan and STILL HAVEN'T FOUND IT!!!
• This year, search tools received a slightly higher rating for responsiveness. Search engine technology cannot yet overcome the gap between some users' wish for more powerful search features and their inability to exploit even the basic features of existing search tools.

• Although equipment upgrades have improved search speed, customers are still struggling with formulating precise searches and finding the results they seek. Analysis of search logs shows that most customers search for general terms (e.g., "grants," "technology," "standards") for which the search engine has trouble returning the most relevant items from the hundreds of thousands of items in the search index. Logs show that few customers avail themselves of the help screens to learn how to search effectively.

  Another issue is that as the amount of material on the web site grows and some of it ages, the difficulty of highlighting the most current information in search results has become a significant problem, which we are addressing by archiving older content and introducing a new sort option called "date relevance."

• Most respondents who chose to comment felt that search results are not precise enough and waste their time with marginally relevant materials, which is consistent with the imprecise search terms the logs show they tend to use.
• I don't have any trouble getting to where I need to search and its so easy to find what I am looking for.
• The search provides so many hits that it is hard to find the information I need sometimes
• Could not find a search capability within the web site.
• Let people know the search button at the top of the screen is for searching within the site.
• The ability to find information using your search engine is pathetic!

	Kind of Information	Avg. Rating* (Rank)			10/16/2001 through 10/15/2002 (last year)	10/16/2002 through 9/7/2003 (this year)
		Cum.	Last Year	This Year
H i g h	Legislation and regulations	2.35 (11)	2.30 (11)	2.38 (1)	Rated highest by Researchers (2.78) Rated lowest by Parents (2.13) Students (2.18)	Rated highest by Managers (2.71) Teachers (2.50) Rated lowest by Researchers (2.17) Parents (2.25)
	Updates on budget, legislation, and activities	2.43 (4)	2.33 (4)	2.35 (2)	Rated highest by Researchers (2.83) Rated lowest by Parents (2.13)	Rated highest by Managers (2.57) Researchers (2.50) Rated lowest by Parents (2.20)
	General guides to the Department of Education and its programs and services	2.37 (10)	2.33 (4)	2.34 (3)	Rated highest by Researchers (2.65) Managers (2.47) Rated lowest by	Rated highest by Researchers (3.00) Managers (2.56) Rated lowest by Parents (2.18)
	Student aid information	2.31 (15)	2.32 (8)	2.34 (3)	Rated highest by Researchers (2.81) Teachers (2.50) Rated lowest by Parents (2.17)	Rated highest by Managers (2.58) Rated lowest by Parents (2.25)
	Announcements of funding opportunities and information about grants and contracts	2.48 (1)	2.39 (1)	2.32 (5)	Rated highest by Researchers (2.86) Rated lowest by Parents (2.24)	Rated highest by Managers (2.77) Researchers (2.67) Rated lowest by Parents (2.16)
	Descriptions of ED funded projects	2.34 (12)	2.26 (17)	2.32 (5)	Rated highest by Researchers (2.73) Rated lowest by Parents (2.14) Students (2.18)	Rated highest by Managers (2.69) Teachers (2.44) Rated lowest by Parents (2.10)
M e d i u m	Descriptions of exemplary schools and programs	2.39 (8)	2.30 (11)	2.30 (7)	Rated highest by Researchers (2.63) Rated lowest by Parents (2.25)	Rated highest by Researchers (3.00) Managers (2.50) Rated lowest by Parents (2.17)
	Conference calendars and announcements of upcoming events	2.31 (15)	2.31 (10)	2.29 (8)	Rated highest by Researchers (2.62) Teachers (2.42) Rated lowest by Parents (2.19) Students (2.20)	Rated highest by Managers (2.55) Rated lowest by Researchers (2.20)
	Full text publications	2.38 (9)	2.30 (11)	2.29 (8)	Rated highest by Researchers (2.75) Teachers (2.45) Rated lowest by Students (2.18) Parents (2.19)	Rated highest by Managers (2.69) Researchers (2.60) Rated lowest by Parents (2.03)
	Directories of information centers, clearinghouses, and technical assistance centers	2.43 (5)	2.34 (3)	2.28 (10)	Rated highest by Researchers (2.86) Rated lowest by Parents (2.26)	Rated highest by Researchers (3.00) Managers (2.50) Rated lowest by Parents (2.03)
	Descriptions of effective and promising practices	2.41 (7)	2.30 (11)	2.26 (11)	Rated highest by Researchers (2.52) Managers (2.42) Rated lowest by Parents (2.21) Students (2.25)	Rated highest by Researchers (2.86) Managers (2.53) Teachers (2.40) Rated lowest by Parents (2.17) Students (2.17)
	Research reports	2.44 (3)	2.35 (2)	2.24 (12)	Rated highest by Researchers (2.74) Rated lowest by Parents (2.17) Students (2.27)	Rated highest by Researchers (2.44) Managers (2.44) Rated lowest by Parents (2.10)
	Publication announcements	2.33 (13)	2.27 (16)	2.24 (12)	Rated highest by Researchers (2.73) Teachers (2.37) Rated lowest by Students (2.17)	Rated highest by Researchers (2.80) Managers (2.58) Rated lowest by Parents (2.09)
L o w	Activities for families, parents, and children	2.26 (17)	2.26 (17)	2.20 (14)	Rated highest by Researchers (2.56) Teachers (2.41) Rated lowest by Students (2.20)	Rated highest by Researchers (2.60) Teachers (2.37) Rated lowest by Parents (2.03)
	Statistics	2.42 (6)	2.33 (4)	2.19 (15)	Rated highest by Researchers (2.83) Rated lowest by Parents (2.13) Students (2.26)	Rated highest by Researchers (2.57) Managers (2.50) Rated lowest by Parents (1.97) Students (2.11)
	Research findings (syntheses and summaries)	2.46 (2)	2.32 (8)	2.19 (15)	Rated highest by Researchers (2.70) Managers (2.52) Rated lowest by Parents (2.14) Students (2.18)	Rated highest by Managers (2.44) Researchers (2.44) Rated lowest by Parents (2.10)
	Press releases	2.32 (14)	2.28 (15)	2.18 (17)	Rated highest by Researchers (2.55) Rated lowest by Parents (2.18) Students (2.18)	Rated highest by Researchers (2.75) Managers (2.40) Rated lowest by Parents (2.12) Students (2.13)
	Speeches and testimony	2.11 (18)	2.20 (18)	2.11 (18)	Rated highest by Researchers (2.59) Rated lowest by Parents (2.08) Managers (2.09)	Rated highest by Researchers (2.40) Managers (2.33) Rated lowest by Students (2.06)
	Total: all categories	2.36	2.31	2.27	Rated highest by Researchers (2.70) Rated lowest by Parents (2.19) Students (2.24)	Rated highest by Researchers (2.63) Managers (2.54) Rated lowest by Parents (2.13)

*  5=very satisfied, 4=satisfied, 3=neither satisfied nor dissatisfied, 2=dissatisfied, and 1=very dissatisfied. Parenthesized number indicates category's rank among 18 satisfaction categories.

• Respondents consider most kinds of information offered to be useful or very useful - an overall cumulative average of 2.36 on a scale of 1 to 3.

• The overall usefulness rating dropped slightly this year from 2.31 to 2.27. Overall the percentage of information types rated useful or very useful dropped from 63% to 54%.

• In this year's responses, researchers and administrators/managers tend to rate higher, while parents rate lower.

• Ratings dropped for 13 of the 19 categories. Ratings rose most for legislation and regulations and descriptions of ED-funded projects. Ratings dropped most for statistics, research findings, lesson plans and teacher guides, and research reports.

• Different audiences value different types of information. The following table lists the highest and lowest rated information categories for the major audience types.

Respondent Audience Type	Rated Highest	Rated Lowest
Administrator/Manager	1. Funding opportunities 2. Legislation & regulations 3. Descriptions of ED-funded projects	1. Speeches and testimony 2. Activities for families 3. Lesson plans 4. Press releases
Parent	1. Conference calendars 2. Student aid information 3. Legislation & regulations	1. Statistic 2. Activities for families 3. Directories of info. centers & clearinghouses
Researcher	1. Exemplary schools 2. Directories of info. centers & clearinghouses 3. General guides to ED	1. Legislation & regulations 2. Conference calendars 3. Student aid information
Student	1. Student aid information 2. Directories of info. centers & clearinghouses 3. Legislation & regulations 4. Descriptions of ED-funded projects	1. Speeches and testimony 2. Statistics 3. Press releases
Teacher	1. Updates on budget & legislation 2. Legislation & regulations 3. General guides to ED 4. Descriptions of ED-funded projects	1. Lesson plans 2. Press releases 3. Speeches and testimony

Question #12 was removed from the survey in September 1998 in response to customer requests to shorten the survey. Analysis of the responses received before then is included in previous memoranda.

Overall response to the fourteen potential new services listed on the survey indicates that we are on the right track. Most of the services that received the highest number of votes have been addressed since the survey began or will be addressed by projects already planned or underway.

In descending order of their popularity in this year's responses, the fourteen new services listed on the survey are:

Kind of New Service	% Vote for (Rank) *			10/16/2001 through 10/15/2002 (last year)	10/16/2002 through 9/7/2003 (this year)
	Cum.	Last Year	This Year
Electronic submission of grant applications ED implemented e-Application, an electronic grant application system <gapsweb.ed.gov/e-app/eaHome.asp>, in 2000.	51.6 (1)	42.9 (1)	43.0 (1)	Rated highest by Researchers (50%) Students (50%) Rated lowest by	Rated highest by Students (59%) Teachers (53%) Rated lowest by Researchers (7%)
Education resource organization directory The Education Resource Organizations Directory (EROD) <www.ed.gov/erod/> was implemented in February 1997, contains information on 4,000+ organizations, and is used 10,000+ times per month.	50.6 (2)	40.2 (2)	40.2 (2)	Rated highest by Researchers (66%) Parents (49%) Managers (48%) Rated lowest by Students (31%)	Rated highest by Teachers (53%) Managers (48%) Rated lowest by Parents (35%)
Electronic submission of student aid applications Student Financial Assistance (SFA) implemented FAFSA on the Web <www.fafsa.ed.gov> in 1997 and improves it each year.	38.2 (8)	33.8 (5)	38.2 (3)	Rated highest by Students (51%) Rated lowest by Managers (19%)	Rated highest by Students (62%) Managers (2.56) Rated lowest by Researchers (13%) Managers (14%)
Search full text of education information across all federal Internet sites The Cross-Site Index <www.ed.gov/help/site/expsearch/index.html?src=ln> was implemented in February 1998. It currently indexes nearly 500,000 files on 200+ ED-funded web sites and another several hundred education-related web sites for Federal Resources for Educational Excellence (FREE) <www.ed.gov/free/>.	43.7 (5)	39.3 (3)	34.1 (4)	Rated highest by Researchers (63%) Teachers (46%) Rated lowest by	Rated highest by Managers (52%) Teachers (43%) Researchers (40%) Rated lowest by Parents (30%)
Electronic submission of publication orders The ED Pubs online publication catalog and ordering facility <www.edpubs.org> was implemented in August 1998 and received outstanding reviews in a government-wide customer service poll in 1999.	48.1 (3)	31.7 (8)	32.1 (5)	Rated highest by Researchers (50%) Teachers (39%) Rated lowest by	Rated highest by Managers (48%) Teachers (40%) Rated lowest by Parents (23%)
Electronic submission of survey responses NCES has implemented electronic submission for all IPEDS (postsecondary institutions) and some library surveys. NCES plans to do more surveys that way in the future.	42.8 (6)	32.3 (7)	30.1 (6)	Rated highest by Researchers (47%)) Rated lowest by Teachers (27%)	Rated highest by Managers (52%) Students (37%) Teachers (37%) Rated lowest by Researchers (20%)
Database/search of published statistical indicators, tables, and charts NCES implemented the Encyclopedia of Education Stats <nces.ed.gov/edstats/> in 1999 to help customers find relevant information in major compendia. Quick Tables & Figures <nces.ed.gov/quicktables/> helps customers find the most current info from Education Statistics Quarterly.	47.0 (4)	34.4 (4)	27.7 (7)	Rated highest by Researchers (69%) Managers (57%) Rated lowest by Students (25%)	Rated highest by Managers (48%) Researchers (40%) Teachers (33%) Rated lowest by Parents (23%) Students (29%)
Search collections of lesson plans and other teacher materials across many Internet sites The Gateway to Education Materials (GEM) <www.thegateway.org> was implemented in March 1998 and currently contains more than 22,000 items from more than 350 member web sites.	36.8 (9)	29.0 (9)	27.3 (8)	Rated highest by Teachers (46%) Researchers (41%) Rated lowest by	Rated highest by Teachers (43%) Managers (38%) Rated lowest by Researchers (13%)
Custom table generator for education statistics NCES has implemented online search and analysis capabilities for several of its major survey data sets <nces.ed.gov/pubsearch/onlinedata.asp>.	42.2 (7)	32.6 (6)	24.9 (9)	Rated highest by Researchers (66%) Managers (52%) Rated lowest by Parents (24%)	Rated highest by Managers (48%) Researchers (33%) Rated lowest by Teachers (20%)
Live "chat" sessions on education topics ED has no current plans for such a capability but will continue to monitor customer interest.	29.0 (11)	25.1 (11)	24.9 (9)	Rated highest by Students (36%) Teachers (31%) Rated lowest by Managers (21%)	Rated highest by Students (35%) Teachers (30%) Rated lowest by Managers (5%) Researchers (20%)
Live "town hall" meetings with Department representatives The modest support for such a capability matches its position on ED's list of planned enhancements.	30.7 (10)	22.7 (12)	23.7 (11)	Rated highest by Teachers (31%) Researchers (28%) Rated lowest by	Rated highest by Students (32%) Rated lowest by Researchers (13%) Managers (14%)
Ongoing moderated discussion areas Web-based discussion forums were implemented, beginning in 1998, to support several working groups and topical discussions at <wdcrobcolp01.ed.gov:8000>. Most are currently dormant.	28.7 (12)	20.2 (14)	23.3 (12)	Rated highest by Students (26%) Rated lowest by Teachers (15%)	Rated highest by Teachers (33%) Managers (29%) Rated lowest by Researchers (13%)
Video transcripts of speeches and presentations by Secretary and other ED representatives In 1998, the Office of Public Affairs (OPA) began sponsoring live and archived video webcasts of news events at <www.connectlive.com/events/deptedu/>.	24.4 (13)	25.4 (10)	20.9 (13)	Rated highest by Students (33%) Rated lowest by Managers (21%)	Rated highest by Students (31%) Rated lowest by Researchers (13%) Parents (15%)
Audio transcripts of speeches and presentations by Secretary and other ED representatives In 1998, the Office of Public Affairs (OPA) began regularly posting short audio clips at <www.ed.gov/news/av/audio/index.html?src=ln>.	23.0 (14)	21.5 (13)	19.3 (14)	Rated highest by Students (26%) Rated lowest by	Rated highest by Researchers (29%) Students (26%) Rated lowest by Teachers (13%)
Total: all categories	38.3	30.8	29.3	Rated highest by Researchers (44%) Rated lowest by	Rated highest by Students (35%) Managers (35%) Teachers (34%) Rated lowest by Researchers (23%)

*  Parenthesized number indicates category's rank among the 14 potential new services.

Customer Suggestions for Improving ED's Internet Services

This year's responses are consistent with trends noted in previous years in respondents' suggestions for improving ED's Internet services in their comments to Questions #14, #15, and #7-10:

• put all ED information online promptly and keep it up-to-date;

• present information in layman's terms, with simple explanations of complicated rules and procedures;

• provide a tutorial or quick tour for first-time visitors to the web site to help them navigate the site;

• provide an orientation to the Department, including its mission, organization, scope, functions, history, and relation to state, local, family, and individual roles in American education;

• provide complete contact information (including email, traditional mail, telephone, and fax) for all Department officials, programs, services, activities, and investigative/enforcement functions; add email addresses to ED staff directory;

• improve responsiveness to customer inquiries submitted via email, web forms, and telephone, particularly for student financial aid;

• improve ease of use for student financial aid services (e.g., obtaining a PIN, submitting and revising FAFSA on the Web, checking status of applications, account balance, payment history, downloading loan forms, etc.);

• make it easier to find specific student aid services, e.g., deferment, forbearance, and all other forms, interest rates, loan consolidation and servicing, customer service contacts, resolution of defaulted loans, etc.;

• provide comparative rating and ranking information about schools, districts, and states for student achievement, teacher performance, etc.;

• use the web to support the full grant lifecycle (application, award, and post-award processes) by:

  • providing an always-current forecast of funding opportunities;
  • maintaining web pages with comprehensive, up-to-date information about each grant program in a consistent format;
  • making it easy for people and organizations to find grants for which they are eligible; and
  • providing links from grant competition pages to other information that could be helpful to prospective applicants, e.g., program information, profiles of previous award recipients, and winning applications;

• offer more information and resources of direct utility to parents and teachers, such as lesson plans, educational materials, advice to help parents help their children learn, and professional development workshops and other opportunities for teachers; and

• provide a portal/gateway to help visitors find more education-related information and services elsewhere, including:

  • funding opportunities and teacher certification requirements at state education agencies,
  • scholarship opportunities,
  • college & university degree programs, courses, and ratings,
  • local agencies and authorities for education, child care, housing, safety, and related issues.

Attachments:

A: ED Internet Customer Survey - Summary of Responses [7 pages]
B: ED Internet Customer Survey - Selected Comments (Rants & Raves) [2 pages]
C: ED Internet Customer Survey - Web form [6 pages]