Lundberg and Cobitz (1999)

The increasing use of technology in counseling over the past generation has mirrored society’s adoption of computers as an integral part of our world. Computer applications have become a common resource which counselors use in a variety of ways in their day to day work. The availability of inexpensive and relatively simple computer networks has been the most recent step in this continuing technological evolution (Sampson, Kolodinsky & Greeno, 1997). As a result, there is a great deal of new conversation in counseling circles regarding the current use of technology, much of it spurred by this surging growth of the Internet, its related networks, and their potential application to the profession.

To evaluate the current status of technology usage in counseling assessment, the Association for Assessment in Counseling (AAC) Executive Council supported a survey of their membership concerning the use of technology in counseling-related assessment. Objectives of the survey were as follows:

b. Determine if there is a need for specific kinds of assessment technologies and what those needs might be.

c. Determine if policies, procedures or ethical standards are in effect for technology usage and what additional standards might be necessary or potentially useful.

The survey was seen as a means of promoting the professional development of AAC and American Counseling Association (ACA) members by enhancing technological knowledge and competence in the areas of assessment, measurement, evaluation, and interpretation. It was also developed as a means of promoting the professionalization of counseling through acquisition of information on current standards and ethical procedures used in technology, in addition to outlining the need for other standards.

A survey of the AAC membership was conducted during the summer of 1997 with a follow-up survey during the summer of 1998.A one page (two-sided) survey form was constructed to coincide with the objectives.Survey questions on specific uses of known technology appropriate to the assessment process were developed following the basic definitions of assessment, measurement, evaluation, and interpretation (Payne, 1992; Vacc & Loesch, 1994).The assessment process of fundamentally gathering and processing data was viewed in relation to current, known computer technologies that accomplish these tasks. The result was a list of fourteen different types of technology potentially used in the counseling assessment process. Survey participants were also asked what other ways they used technology in counseling-related assessment.Three questions on policies, procedures, and ethical standards were formulated. Gathering information on effectiveness of technology and future needs was considered potentially valuable to the counseling profession; three questions were asked covering these areas.The resultant survey form is located at www.ncat.edu/~schofed/aac.

The survey and follow-up were distributed via the association newsletter, AAC Newsnotes, to the entire membership (all recipients of the newsletter). Readers were given the opportunity to respond in one of four ways: by business reply mail (a business reply envelope was included with the paper survey form in the newsletter), by Internet Website, by e-mail, and by fax.The two mailings went to an average of 2,130 readers, and 153 different individuals responded. This resulted in an overall response rate of 7.2%.

Of the 153 respondents, 56% were men and 44% were women. The majority of the respondents were Caucasian-American (86%), with other ethnicities represented as follows: African-American (3%), Hispanic-American (3%), Native American (1%), Asian-American (1%), and other (3%).Four individuals (3%) did not respond to ethnicity. Among the survey respondents, about half (48%) worked in academic settings (public/private schools, higher education), and the other half (52%) were employed in other settings (private practice, business/industry, consulting, test publishing, etc.). With regard to highest degree attained, almost half of the respondents (48%) held doctorates, and the rest (52%) indicated specialist, master’s or other degrees.

To assess the representativeness of this sample, the demographic characteristics of the respondents were compared with those of the AAC membership (provided by ACA as of the summer of 1997 and shown in Table 1).AAC membership was 45% male and 55% female. Members were primarily Caucasian-American (85%) with 6% African-American, 4% Hispanic-American, 2% Asian-American, 1% Native American, and 2% other ethnicities. With regard to employment, 45% of AAC members were employed in academic settings, and 55% were employed in other settings. Thirty-one percent held doctorates. Four chi-square goodness-of-fit tests were performed to determine the similarity of the sample to the membership. A Bonferroni correction was applied to control for potential Type I errors due to the use of multiple tests. Results indicated that for ethnicity and job setting, the sample matched the AAC membership. The sample differed from the membership with regard to gender (c²= 7.31, df = 1, p < .01) and highest degree (c² = 20.96, df = 1, p < .001).

Eighty-two per cent (126) of the survey participants responded by business reply mail.The remaining 18% (27) of the respondents replied via one of the electronic methods with Internet Website being the most popular electronic response (12% of total respondents). The relatively small percentage of individuals using electronic methods may indicate that the familiar method of responding by pencil and paper is still regarded as the easiest way to participate. Survey participants were asked to indicate which of the fourteen types of technology they used in the counseling assessment process. Table 2 outlines each of those types of responses by gender, highest degree (doctorate or non-doctorate), and work setting (academic or private practice/other)

A majority of survey participants in each category noted that they used the Internet for assessment. In addition, every category of respondents indicated that searching for data through the Internet was their most common method of technology use. With most types of technology, a larger percentage of responding males indicated usage than the corresponding percentage of females. Likewise, with most types of technology, a larger percentage of responding doctorates indicated usage compared to non-doctorates, and a larger percentage of academics indicated usage than those in private practice or other settings.

Survey participants were asked to list other ways that they used technology in counseling-related assessment, and 42% (65) responded to this question. Many of those inputs indicated non-assessment uses of technology, but several listed assessment methods currently in existence and not specifically covered by the survey.Use of computerized scoring systems other than scantron or bubblesheet was indicated. Creating databases of assessment results was a potentially powerful tool being developed. A comprehensive assessment program incorporating multiple computerized instruments, patient tracking with pre- and post-treatment measures, along with computerized biofeedback assessment is one type of development currently being used which could see more widespread and effective usage in the future. The use of video and audiotapes in assessment was noted, as was the use of CD-ROM to gather data and the use of e-mail or fax to transmit data.

Respondents were asked to identify which of the methods were most effective and most cost-effective. Although these two questions lacked specific criteria to measure effectiveness, 42% (64) responded to the effectiveness question in a discriminating manner and 38% (58) responded to the cost-effectiveness question in a similar way. Some individuals responded that all forms of technology were effective. This type of response was eliminated.

Among the respondents to the effectiveness question, 34% (22) noted Internet data searches as among the most effective types of assessment-related technology. Although this was the most common response, some individuals noted the unevenness of Internet data.Electronic testing and the use of scantron or bubblesheets were noted as effective uses of technology. Several individuals noted that effectiveness is often a function of usage; the more frequently technology is used, the more effective it generally becomes.

The use of both desktop-based spreadsheets (e.g. Excel, Lotus) and mainframe-based spreadsheets (e.g. SAS, SPSS) was considered effective, and several individuals commented on the usefulness of desktop-based SPSS (Version 7.0 for Windows). The term “spreadsheet” (with appropriate examples) was chosen as a concise term, although it is not a perfect descriptor of all these statistical applications.

Of those responding to the cost-effectiveness question, 50% (29) indicated Internet data searches as among the most cost-effective types of technology used in the assessment process. As in the previous question regarding “simple” effectiveness, Internet usage was the most common response.

Survey participants were asked what ethical issues/dilemmas they had encountered and were given four choices for responding: a) I have encountered no ethical issues or dilemmas in the use of technology, b) access to information, c) intellectual property (plagiarism), and d) others (please list). Seventy-nine percent (121) of those returning the survey responded. Of those answering this question, 55% (67) indicated they had encountered no ethical issues, 30% (36) reported experiencing problems with access to information, 14% (17) noted issues of intellectual property, and 22% (27) identified other ethical issues. Of those commenting on other ethical issues, most addressed non-assessment areas (e.g. distance counseling). Several respondents reported experiences with authentication of information (e.g. production of fraudulent transcripts, inaccurate Internet data). Some noted the problem of response bias through electronic testing. Overall, there was considerable and widespread concern with the issue of ethics in technology.

The survey addressed standards and policies in the use of assessment-related technology by asking two open-ended questions. First, participants were asked what standards, policies or procedures they used, and second, what (other) ones would help in their practice. Approximately 51% (78) of the respondents made specific comments with regard to the first question.

The single most common response regarding currently used standards was to follow already published ethical guidelines. Various sets of standards or codes of ethical practice were indicated by 49% (38) of those responding to this question, the most common response being to use the ACA standards and the American Psychological Association (APA) standards. The next most common response was to use some form of local access control as a policy or procedure, noted by 26% (20).These ranged from password protection when accessing computer information to physically locking up computer disks containing sensitive information.

A variety of other standards or procedures were indicated including general ethics, checking the credibility of Internet sources, adherence to copyright laws, and simple scheduling of hardware use. Several individuals responded with a comprehensive series of standards. These generally fell into two categories: standards of practice for counseling assessment or policies for computer management (hardware and software procedures). An example highlighting standards of practice was the integrative use of a district or institutional technology agreement, joint evaluation standards, personal ethical standards, combined with testing program standards. One example focusing on computer management emphasized strict licensure constraints, policies on e-mail and Internet usage, and centralized installation and procurement of hardware and software.

When asked what other standards, policies or procedures would be helpful, 22% (33) of the survey participants made specific comments, and the responses generally fell into two categories. Of those responding, 55% (18) indicated the need for more standards (e.g. Internet policies, Joint Commission assessment standards, ACA technology standards, clarification of response bias with electronic surveys and testing, standards for the ethical use of technology in general). Several respondents noted the need for access controls that limit an administrator’s ability to enter and manipulate subordinates’ computer accounts. A second set of inputs, 30% (10 individuals) of the respondents, asked for more procedures in the areas of electronic testing, comprehensive data integration, video conferencing, and for better technology in general. Some individuals suggested more technology education and workshops.

When asked what other types of technology respondents viewed as being needed in the area of assessment, 38% (58) provided narrative responses. Those inputs generally fell into three broad categories: a) increased or improved quality of assessment with technology, 2) increased ease of use of technology, and 3) ethical issues. A few responses could not be categorized or understood.

With the most common type of response, 45% (26) of those answering this question were concerned with quality of assessment technology. Included were timely processing of information, increased data integration, virtual reality simulations, integrated test batteries, fast and user-friendly desktop-based statistical programs, career assessment software, computer-adaptive testing, integration of methodologies, inexpensive scoring software, and testing/assessment programs allowing clinical input rather than strictly data input.

In the next most common set of responses, 34% (20) were concerned with ease of use of technology. Within this group were voice recognition software, video conferencing, assistive technology, more Internet opportunities for data collection and assessment, smaller and more accessible assessment units (e.g. hand-held instruments), more extensive availability of e-mail, and large scale Internet assessment.

The last category, ethical issues, consisted of 8 inputs, (14% of those responding). Included were e-mail ethical guidelines, insuring human interaction with technology, restricting clinical assessment to those individuals who are qualified, and security measures for electronic test items.

Although it was possible to group most of these responses into three broad categories, this portion of the survey was notable for the diversity and specificity of inputs. Few of the suggestions for needed technology were noted by more than one or two respondents. Technology was viewed as potentially able to meet widespread needs in the area of assessment.

Respondents to this survey are using technology in counseling-related assessment in a wide variety of ways. Existing professional standards are being applied in the technology area, individuals are formulating their own sets of guidelines, and they desire additional standardized procedures. Ethics in the use of technology is an issue of broad concern, with access to information being the most commonly identified problem area. Even with its lack of standardization, the Internet has become a prominent technological tool in assessment.

Approximately 76% of responding doctorates were male. The strong response to this survey by males, and particularly by those holding doctorates, seems to indicate that individuals holding the terminal degree are more apt to respond to these types of surveys. However, the percentages of responding females and males using the Internet for assessment purposes were very close. Perhaps this suggests that females are moving more readily into this newer form of technology.

The expectation that counselors will make good use of technology is grounded in the modeling of realistic, effective, and ethical uses during their primary educational experience (Stevens & Lundberg, 1998).Increasing the awareness of technology uses, followed by skill training and workshops on integrating technology into effective professional practice, is needed.In addition, developing technology expertise and effective training within counseling curricula is a next step.

Payne, D. A. (1992). Measuring and evaluating educational outcomes. Macmillan: New York.

Sampson, J. P., Jr., Kolodinsky, R. W., & Greeno, B. P. (1997). Counseling on the Information Highway: Future possibilities and potential problems. Journal of Counseling & Development, 75, 203-218.

Stevens, D. T, & Lundberg, D. J. (1998). The emergence of the Internet: Enhancing career counseling education and services. Journal of Career Development, 24(3), 195-208.

Vacc, N. A., & Loesch, L. C. (1994). A professional orientation to counseling (2nd ed.). Muncie, IN: Accelerated Development.