Abstract
284
career-related websites were evaluated by 68 graduate students (54
females, 14 males) enrolled in two sections of a Master’s level Career
Development course. Results suggest that overwhelmingly, students rate
these websites positively, with little discrimination among sites being
evidenced. Site organization, information provided and links were the
most often factors considered. Students’ knowledge of specific web-sites
increased nearly 100% over a semester. Limitations and suggestions for
future research are provided.
The Internet provides a myriad of sites
related to careers. Graduate students pursuing a degree in counseling or
career counseling often learn about these sites during their academic
and professional training. This information is essential for graduate
students because the Internet greatly increases their access to
important information (McCarthy, Moller, & Beard, 2003). In addition,
research indicates that students have access to and are willing to use
the Internet (Zalaquett & Sullivan, 1998) for self-help; and to support
counseling-related activities (Clark & Stone, 2002;
McCarthy et al.,
2003). Career counselors who are successful in integrating the Internet
into their interventions are of help even to clients who conduct their
own career searches online (Noll & Graves, 1998); and graduate students
who have been exposed to specific career websites feel more confident
and competent to assist clients in Internet searches (Edwards,
Portman, & Bethea,
2002).
Using the Internet is an important task for
career counselors to undertake because it presents many
 opportunities
for supporting career counseling activities on a global level (Gore & Leuwerke, 2000;
McCarthy et al., 2003), including online career
assessments (Oliver & Whiston, 2000), increased occupational information
(Lock & Hogan, 2000), specialized career web-sites for populations such
as women and minorities (Knouse & Webb, 2001), virtual job fairs (Miller
& McDaniels, 2001), job search strategies (Reile & Harris-Bowlsbey,
2000), as well as job search difficulties for today’s job seeker (Pelling,
2002).
Most of the authors’ anecdotal experiences
with educating graduate students suggest that the counselors-in-training
find learning about career-related websites beneficial for their
professional development and to better help their clients. Similar
observations have been reported in the research literature in the field
(Edwards et al., 2002;
McCarthy et
al., 2003). Moreover, the authors have observed a general tendency
of graduate students to rate Internet-based career related sites
positively. This tendency was also found by
McCarthy et al. (2003), as
well as
Layne and
Hohenshil (2002).
McCarthy et al. (2003) examined
reactions about using Internet sites from graduate counseling students
(N=14) who incorporated Internet career sites as a component of career
advising with undergraduate clients, as well as the reactions of the
undergraduate clients (N=20). Both groups rated the experience very
positively, saw the merit of using Internet sites as a part of career
counseling, learned new information as a result, and would use the
Internet sites in the future for career-related activities.
Layne and Hohenshil (2002) found that out of
891 statements from 74 graduate students enrolled in an online course
about the positives and negatives of using the Internet in career
counseling, 60% of those were positive. Thematic analyses of these
positive statements included: increasing client access to information;
helping clients meet career and educational goals; making the job search
easier; increasing access to career assessments; and saving time.
Disadvantages included: unequal access to computers/internets among
clients; overwhelming amounts of information; and outdated, incorrect,
inappropriate or biased information.
How do graduate students evaluate career
sites when given specific factors to consider? Forty such sites were
recently reviewed in a special issue of the Career Planning and Adult
Development Journal (2002). These sites were assigned to and
reviewed by various professionals in the helping field, with five sites
being reviewed by graduate students. The evaluation of these sites
included an in-depth review on several factors, and a final evaluation
of two thumbs up (Excellent; N=23), one thumb up (Fair; N=17), or no
thumbs up (Poor; N=1). The majority of the graduate students (four out
of five) rated their assigned sites very favorably with two thumbs up,
with one student giving their site one thumb.
The tendency of graduate students to rate
Internet-based career related sites positively suggests, on one hand,
that they see the benefit of this information and are not afraid of this
technology. On the other hand, it suggests that the students may have a
tendency to see career related sites as positive across the board. Much
of the research in this area has provided support for our first
suggestion; however, little research has attempted to explore our second
suggestion. This is troublesome because the existence of a
career-related site doesn’t guarantee that it is a quality site that
should be used with clients. Professional organizations such as The
National Career Development Association (NCDA; 1997, Guidelines for
Use) and the Association of Computer-Based Systems for Career
Information (ACSCI;
2002, ACSCI Standards) have identified several
factors that both counselors and clients should consider when evaluating
and determining which career web sites to use with or recommend for
clients.
Using the Internet to conduct and/or enhance
career counseling is a current topic of much discussion among career
counseling professionals (NCDA,
1997;
Boer, 2001). Given that empirical
studies on career-related Internet sites are rare (Gati, Kleiman, Saka &
Zakai, 2002), the
NCDA Ethics Committee (1997, Need for Research and
Review), have stated that “…it is mandatory that the career counseling
profession gain experience with this medium and evaluate its
effectiveness through targeted research.” Career counselors have been
cautioned, however, to be aware of potential ethical dilemmas regarding
Internet use with career counseling (Reile & Harris-Bowlsbey, 2000;
Sampson, Kolodinsky & Greeno, 1997), as well as those associated with
many online career assessments’ psychometric limitations (Oliver & Whiston, 2000;
Oliver & Zack, 1999;
Sampson & Lumsden, 2000)
Most of the current research on Internet
awareness and utilization by
 graduate students focuses on the possible
benefits or barriers for using this resource in career counseling, but
little efforts have been made to determine how graduate students “see”
these sites. This is an important issue because understanding how
counselors-in-training perceive career-related web sites will provide
relevant information for course structuring and delivery. More over,
from an academic perspective we need to ensure that graduate students
have the capacity to critically evaluate Internet career resources. The
purpose of this study was to explore how graduate students perceive
career sites, how graduate counseling students’ awareness of
career-related websites change over a semester, and how they view and
ultimately evaluate specific career-related websites.
Methods
Participants
284 career-related websites were evaluated by 68 graduate students (54
females, 14 males) enrolled in two sections of a Master’s level Career
Development course. The students participated in the study in two
different semesters. Forty students participated in the first semester
(12 males, 28 females), and 28 participated in the second semester (2
males, 26 females). These students were enrolled at a large southeastern
university located in a metropolitan area of about 2 million persons.
Participants agreed to allow the results of a class assignment to be
used for research purposes.
Procedures
Students enrolled in a graduate level Career
Development course were invited to participate in this study on Internet
experiences and opinions of counseling students. Students who agreed to
participate completed a general questionnaire prior to evaluating
randomly assigned career-related Internet sites that were available on a
main website used for the course,
http://careerresource.coedu.usf.edu/. Some sample sites included The
Career Key (http://www.careerkey.org/english/),
the ONET Consortium (http://www.onetcenter.org/tools.html),
Working Wounded (http://www.workingwounded.com/),
and éResume (http://www.eresumes.com/).
The purpose of this questionnaire was to obtain a baseline of existing
knowledge of career-related websites. After the general questionnaire
was completed, students were assigned three to five (depending on
semester) websites to review, and asked to answer general questions
about their opinion of each site. Several weeks after completing the
assigned evaluations, students completed a final questionnaire about
their current knowledge of specific career sites to assess for gains in
knowledge. Participation was voluntary, and students were informed that
they could discontinue their participation at any time without penalty.
Open-ended answers were analyzed for themes. Means and percentages were
also calculated.
Instrumentation
Participants were asked to complete three questionnaires designed by the
investigators to address the specific research questions outlined for
this study. The first questionnaire included demographic questions and
asked them to identify specific sites/urls for five specific career
areas, including: self-assessment, career information, career searches,
job preparation and for K12 students.
The second questionnaire was a generic
evaluation form to be used by the participant when evaluating assigned
career Internet sites (See
Table 1 for Questionnaire). These sites were identified by the researchers
via Internet searches and were not rigorously screened for meeting
NCDA/ACSCI
standards. That is, while some of the sites were commonly known and used
by career development professionals (such as the online
Occupational
Outlook Handbook or
O*Net), others were not (such as
wetfeet.com or
jobfind.com).
Questions were related to issues such as
what was liked most/least about the site (open-ended), how attractive,
trustworthy and expert the site appeared to be (on a scale of 1 to 5,
with one being “not at all” and five being “completely”), whether the
person would recommend the site to a friend (and reasons for or against
recommendation), and the relevancy of the activity of evaluating career
sites given the course content. 284 sites were evaluated. A final
questionnaire, given at the end of the semester asked students to
identify specific sites/urls for the five specific career areas
mentioned above.
Data Analysis
Responses to the open-ended questions were
analyzed for common themes for students in both semesters. Themes were
identified by the first researcher and confirmed by the second
researcher. A graduate assistant then sorted the responses into the
pre-determined themes. Means of an overall rating of the assigned sites
were calculated for the students in the first semester. Based on that
information, the authors asked for more specific components with respect
to rating for students in the second semester. Mean ratings and
percentages of the site categories were calculated with respect to
attractiveness, trustworthiness, expertness and relevancy of evaluating
specific career-related websites given the course content for those
students in the second semester.
Results
Students listed a total of 82 sites on the
first questionnaire. Comparison of results between the first and final
questionnaires revealed a dramatic increase in student awareness of
sites/urls in each career area, with the most dramatic increases seen in
career information and job preparation. The total number of all sites
listed increased from 80 on the first questionnaire to 156 on the final
questionnaire, for an almost 100% increase in knowledge.
In the first semester of our study, we asked
students to provide a global assessment of the sites on a 1 to 10 scale.
Students rated the sites highly, with a mean of 6.92 (SD=2.5). Based on
this information, we were more specific in the next semester, asking
students to rate the sites on aspects of trustworthiness, expertness and
attractiveness. Mean ratings of individual site evaluations ranged from
an average of 3.2 to 4.5. Ratings were similar for each site with
respect to trustworthiness (M = 3.57, SD=.88), expertness (M
= 3.57, SD=.88) and attractiveness (M = 3.31, SD=.72). In
addition, students in this second semester rated the activity of
evaluating specific career-related websites as an assignment in a career
development course highly (M = 4.19, SD=.98).
Thematic
Analyses
240 comments
were given about what students liked most about the sites. Common themes
of what students liked most about the Internet sites they reviewed
included: the information provided (N=104; 43%), the
organization/attractiveness of the site (N=95; 40%) and the
number/quality of links provided (N=41; 17%).
245 comments were given about what students
liked least about the sites. Common themes of what participants liked
least about the Internet sites they reviewed included: “nothing” (N=83;
34%), organization/look of the site (N=77; 31%), if the site was limited
in it’s applicability (“didn’t have what I was looking for,” or “only
for a special group,” or “only did” a specific feature like resume
posting; N=42; 17%), advertisements/pop-ups (N=18; 7%), cost or having
to create an account and use personal information (N=15; 6%), inactive
links (N=6; 3%), and a question of who’s running the site (N=4; 2%).
Participants were also asked about how
user-friendly the sites were. User-friendly sites (N= 213; 81%) were
characterized as being well-organized (N=165; 79%), appropriate/working
links (N=18; 8%), useful information (N=12; 6%), and “n/a” (N=14; 7%).
Non-user-friendly sites (N=37; 14%) included poor organization (N=36;
72%), cost or having to set up an account/divulge personal information
(N=7; 14%), inactive or inappropriate links (N=5; 10%) and miscellaneous
comments (N=2; 4%). 5% of comments were classified as either n/a or
maybe.
Of the 284
sites reviewed, 260 received ratings on the recommendation factor (some
students did not indicate whether or not they would recommend the site
or not). 198 (76%) of the sites would be recommended by the students to
a friend. 212 positive comments were made on why a student would
recommend a site. Themes included: organization of the site (N=165;
78%), links (N=18; 8%), “not applicable” (N=17; 8%) and information
provided (N=12; 6%). 52 sites would not be recommended, with 19 negative
reasons given, including that the site was not helpful or did not
provide useful information (N=5; 26%), it was not user friendly and
lacked organization (N=4; 20%), there were better sites out there (N=3;
16%), the cost to use the site (N=3; 16%), they felt uncomfortable using
the site (N=2; 11%) and because there were no or inactive links (N=2;
11%).
Discussion
The results of our study confirm what the
authors’ observation and previous studies (Layne & Hohenshil, 2002;
McCarthy et al., 2003) seemed to indicate with respect to graduate
students’ ratings of career-related websites. Graduate students
overwhelmingly rate these sites as positive in general, and also
specifically with respect to trustworthiness, attractiveness and
expertness, suggesting minimal discrimination among ratings. Given that
these sites were not screened for meeting the majority of
NCDA/ACSCI
standards, we might have expected some differences among ratings for the
sites, which did not occur. When analyzing what students paid attention
to when evaluating sites, how sites were organized (including
attractiveness factors) was consistently of greatest or second greatest
importance to them.
The lack of complexity utilized by students
in evaluating these websites is troubling.
NCDA and
ACSCI have put forth
several factors that should be considered when evaluating career-related
websites for use with clients. Table 1 lists these criteria and provides
columns for evaluating sites based on these guidelines/standards.
Graduate students may be unaware of the
NCDA/ACSCI criteria, and thus
future studies might explore how the introduction of such criteria
affects students’ evaluations of career-related websites. Sites that are
recommended or used with clients that do not meet the
NCDA/ACSCI
criteria might result in increased client frustration due to factors
such as lack of technical assistance or an unclear format/vocabulary and
potential damage. An example of this would be a client making a career
choice based on the results of an online career inventory with limited
(or non-existing) reliability and validity data, such as a site that
provides career options based on a series of color preferences that the
user selects.
Student knowledge of sites increased dramatically after engaging in the
evaluation of three career-related sites.
Clark and Stone (2002)
reported similar findings in anecdotal data from counseling students
that reflected that by engaging in online assignments that utilize
specific websites, they became aware of and comfortable using such sites
with future clients. If students can identify specific sites, perhaps
they will be more likely to use these sites with clients. Future studies
should focus on factors related to clients’ use of career-related
websites, as well as examining whether knowledge of career-related
websites as a graduate counseling student actually impacts future use
with real clients (versus hypothesized ones).
Students found this online assignment very
relevant to the course content. As instructors, the gains in student
knowledge of specific sites are very encouraging, while we find the
complexity of student evaluations somewhat discouraging. Identifying
sound pedagogical strategies for developing a more constructively
critical eye toward websites among students is also a topic for future
studies.
Limitations of this study include a small
sample size with limited variability in gender and ethnicity, as well as
limited variability in self-rated skills and motivation, thus limiting
generalizability. However, as an exploratory study, we believe the
results are consistent enough to provide a foundation upon which other
studies might build. Future research might seek to involve students from
various universities, with varying levels of motivation and skill. In
addition, studies might examine how website evaluations differ for
specific sites as compared across differing groups, such as clients,
graduate counseling students, counselors and career counseling
researchers. Increasing counseling students’ knowledge of career-related
websites, and enhancing the complexity of how they evaluate such sites
will likely yield multiple benefits to our profession, as students
become more sophisticated in their use and recommendation of sites.
Clients also stand to benefit as their counselors can make
recommendations of the best sites, and even provide evaluation criteria
such as those identified by
NCDA/ACSCI to clients who enjoy or regularly
use the Internet as a means to supplement their counseling experience.
References
The Association of Computer-Based Systems
for Career Information (2002). ACSCI Standards. Retrieved May 15,
2003
from
http://www.acsci.org/acsci_pubs1.htm
Boer, P. M. (2001). Career counseling
over the Internet.
Mahwah, NJ: Erlbaum.
Clark, M. A., & Stone, C.
B. (2002). Clicking with students: Using online assignments in counselor
education courses.
Journal of Technology in Counseling, 2(2). Retrieved from
http://jtc.colstate.edu/vol2_2/clarkstone.htm
Edwards, Y. V., Portman, T. A. A., & Bethea, J.
(2002). Counseling student computer
competency skills: Effects of
technology course in training. Journal of
Technology in Counseling, 2(2). Retrieved from
http://jtc.colstate.edu/vol2_2/edwards.htm
Gati,
I.,
Kleiman, T., Saka, N., & Zakai, A. (2002). Perceived Benefits of Using
an Internet-Based Interactive Career
Planning System. Journal of Vocational Behavior. 62,
272-286.
Gore, P. A. Jr., & Leuwerke, W. C.
(2000). Information technology for career assessment on the Internet.
Journal of
Career Assessment, 8(?), 3-19.
Knouse, S. B., & Webb, S. C. (2002).
Virtual networking for women and minorities. Career Development
International,
6(4),
226-28.
Layne, C. M., & Hohenshil, T. H.
(2002). Graduate students view the use of the Internet. Career
Planning and Adult
Development Journal,18,
150-156.
Lock, J. D., & Hogan, R. (2000).
Expanding the focus of career assessment. Journal of Career
Assessment,8(4), 411-17.
McCarthy, C. J., Moller, N., & Beard,
L. M. (2003). Suggestions for training students in using the Internet
for career
counseling. Career Development Quarterly, 51, 368-382.
Miller, K. L., & McDaniels, R. M.
(2001). Cyberspace, the new frontier. Journal of Career Development,
27(3), 199-206.
National Career Development Association.
(1997). NCDA Guidelines for the Use of the Internet for Provision of
Career
Information and Planning Services. Retrieved May 15, 2003, from
http://www.ncda.org.
Noll, C. & Graves, P. (1998).
Technology and educational support offered by university career centers.
Journal of Career
Planning and Employment, 58,
20-24.
Oliver, L. W., & Whiston, S. C.
(2000). Internet career assessment for the new millennium. Journal of
Career
Assessment, 8(4), 361-69.
Oliver, L. W., & Zack, J. S. (1999).
Career assessment on the Internet: An exploratory study. Journal of
Career
Assessment Fall, 7(4), 323-56.
Pelling, N. (2002). The use of technology in career counseling.
Journal of Technology in Counseling, 2 (2).
Retrieved from
http://jtc.colstate.edu/vol2_2/pelling.htm
Reile, D. M., & Harris-Bowlsbey, J-A.
(2000). Using the Internet in career planning and assessment. Journal
of Career
Assessment, 8(1), 69-84.
Robinson, N. K., Meyer, D., Prince, J. P,
McLean, C., & Low, R. (2000). Mining the Internet for career
information: A model
approach for college students. Journal of Career
Assessment, 8(1), 37-54.
Sampson, J. P. Jr., Kolodinsky, R.
W., & Greeno, B. P. (1997). Counseling on the information highway:
Future
possibilities and potential problems. Journal of Counseling and
Development, 75, 203-212.
Sampson, J. P., Jr., & Lumsden, J. A.
(2000). Ethical issues in the design and use of Internet-based career
assessment. Journal of Career Assessment, 8(1), 21-35.
Stevens, D. T., & Lundberg, D. J.
(1998). The emergence of the Internet: Enhancing career counseling
education and
services. Journal of Career Development, 24(3), 195-208.
Zalaquett, C. P., & Sullivan, J. R.
(1998). Counseling center help screens: Students' use and evaluation of
an Internet-based program. Journal of College Student Development, 39(5),
494-98.
Authors' Biography
Debra S. Osborn is
an Associate Professor of Counselor Education in the Department
of Psychological and Social Foundations at the University of
South Florida. She serves as the program coordinator for Career
Counseling. She can be reached at:
osborn@coedu.usf.edu
Carlos P. Zalaquett is an Assistant Professor of Counselor
Education in the Department of Psychological and Social
Foundations at the University of South Florida. He serves as
the program coordinator for Community Counseling. He can be
reached at:
zalaquet@tempest.coedu.usf.edu
|
