Source Evaluation and Search Strategies

Contents

General Source Evaluation
Select a Topic
Finding Information
Make Good Use of Resources for Finding Sources
Evaluation Refined
Evaluation Refined - The Tangled Web

General Source Evaluation

This module will help you understand basic concepts related to locating and evaluating information for research projects. By exploring the different types of sources and the criteria and questions for determining the quality of a source, you will be better prepared to conduct a college-level research literature review.

In addition to a topic, the first thing you have to know is the kind of information you are seeking for your research project. Establishing clear objectives for your research process is paramount. For instance, are you interested in "just the facts?" In that case you may be looking primarily for statistics. Or are you looking for rich descriptions or eyewitness accounts of events or experiences? These objectives will lead you in a different direction from a statistical analysis of your topic.

Read the dialogue from a meeting of these freshman virtual students to gain some hints on things to think about before you begin significant searching and source evaluation. These processes, along with a clear goal for your research, will help you filter the sources you uncover more easily, quickly, and efficiently.


Jose: We've been working on things like time management, stress management, listening skills, and note taking skills, but we haven't talked much about how to do research for a paper. Have any of you had to write a research paper for any of your Penn State classes yet? Or do you have much experience from high school with writing papers ?

Deena: That's a good question, Jose. I do have some experience with writing papers in high school, but that was so long ago! Things have changed a lot since then. I think some of the basic skills will still apply, but I don't know anything about how to evaluate informaiton from the Internet, or even how to cite Web sites. And finding things at the library--that's still a mystery to me.

Brian: Yeah, there are so many resources at the library that sometimes I get so overwhelmed that I just give up!

Jose: These are exactly the kinds of things I need to know before I start working on my history paper. How can I find out what resources are available to use for finding information, and then, how can I learn to use them?

Sage: Have you tried asking the librarian for help? There are also some tutorials on the Web at the library Web site. Have you tried doing those? Or you could take one of the seminars the library offers to learn about the different resources.

Jose: My freshman seminar instructor did tell us about the tutorials and seminars, but I haven't taken time to do any of them yet. You're right Sage, there are things I should do on my own to learn how to use the resources available at the library before I start complaining about how confused I am.

Heather: I like to use search engines on the Web to find information for papers I have to write.

Brian: Now, we are right back where we started. We know we need to do research and that there are a lot of resources available, but how do we know which resources to use and how do we get started?

Heather: First I think you have to identify your topic. If you don't know what your topic is, you won't know which resources to use.

Brian: I have to write a paper for my freshman seminar about what it's like being a freshman in college and compare my experience here at Penn State with what it would be like for a student at a small, private college.

Jose: OK, if we use Brian's topic as an example, can you help me figure out what I would need to do to get started finding information? Brian can use his own experiences to write about what it's like to be a student at Penn State, but where should we look for information about freshmen at other colleges?

Heather: It sounds like Brian's topic could be summarized as the freshman college experience.

Sage: OK. Now that we have a topic, we can develop the steps for our research plan.


Tip: Here are two resources to help you learn to use the on-line library catalog at Penn State.

Quick Guide to Searching in the PSU on-line Catalog:
http://www.libraries.psu.edu/instruction/CAT_guide.htm

On-line tutorial (Flash) for the PSU on-line catalog:
http://www.libraries.psu.edu/instruction/cat_tutorial/index.html


Select a Topic

It's a good idea to think about your project before you do significant searching and source evaluation. Having a clear goal for your research will help you filter the sources you find more easily, quickly, and efficiently. Keep these three things in mind as you work on identifying your topic:


Then, as you think about your topic, ask yourself the following questions.

Tip:  To get started on a research project you need to state your topic and identify the types of information you are interested in finding. Try stating your topic in the form of a question or questions to help you focus your research.


Jose: So if the topic is, "the freshman college experience," what kinds of information do we need to find?

Brian: Well, I would like to try to find some magazine articles that talk about sports and other activities at small private colleges. I think that will give me an idea of some of the things that are different about going to a large public university rather than a small private college. I would also be interested in finding diaries or articles written by students.

Sage:   OK. How should we do that?

Brian:   Go to the library and do some research?

Sage:   Randomly?

Brian:   No, I think we need to get some background information on the topic first. Otherwise, we'll just be stumbling around. I think I need to know the names of some small private colleges and other information so I can focus my search.

Deena:   Yeah. A lot of time would be wasted without first getting some background information. You need enough general information about the topic so you can decide where to focus your research. I think it would help if you wrote down a couple of questions you want to find the answers to before you start to search for information.


Finding Information

Your background search should provide you with a large number and variety of sources. You should do some, or all, of the following background work as you start to collect information about your topic:

Tip: Check your topic words against a thesaurus or subject-heading list to get additional ideas of words to use to search for resources.

The Preliminary Evaluation Process

As you start to research a topic it is normal to make some more-or-less instant decisions about the value of the resources and materials you find. Here are some primary evaluation questions that you can use to determine whether the resources you found will be valuable to you or not. Much of this information can be obtained from bibliographic information, tables of contents, or citations you have found. Remember, at this point you are only trying to get an overview of the topic, not gather specific facts or information.

Currency


Relevancy


Authority


Clarity

Tip: Ask yourself what type of source will be the most useful and most credible for providing information on your particular research topic.   With this question as an overarching framework, you will be able to make some preliminary decisions about what materials are scholarly, relevant, and reliable.

Activity 1:

In a sentence or two, describe why you think these four areas are important when doing scholarly research. Use the "My Notes" tool on the right side of the tool bar at the top of the page to type your answers.


Activity 2:

Complete this quiz (ANGEL Quiz) to see what you have learned.


Make Good Use of Resources for Finding Sources

Brian: OK. I understand that I need to start by writing down some questions I want to find answers to and by searching for background information. Then I need to carefully evaluate the information I find to figure out if it will be good to use in my paper. But I still don't understand how to start searching. How do I go about finding sources at the library?

Sage: I think the best way to start is by identifying some keywords to use when searching the databases. If we know specific book titles, or authors, we can search using those names too, but since we are only just starting to find information about this topic, keywords are probably the best way to get started.

Brian: So should I just go to the library and ask the librarian to help me?

Sage: That's a good idea. I get questions all the time from people I work with about how to find something with a Web search.

Heather: I think that using the library and the Web are very similar today.

Deena: I don't. Well, maybe it's similar because many libraries are accessible from the Web, but they're certainly not identical.

Jose: With the library, you primarily use databases and electronic indexes to find books and articles or connect to the library's on-line access facility. When searching many of the databases at the library you often need to know the name of the book or article, and the author's name. With the Web, you use search engines to search more broadly.

Brian: No matter which one we use, I think we still need to identify the keywords before we can get started.


Use of the Library

For the most part, you will be using databases and electronic indexes to find books and articles in the library, or you will be connecting to the library's on-line access facilities.

Tips


Tip: Here are some links to style manuals and references:

Internet help on APA and Chicago style formats can be found in many places. One place to give APA style is:   http://www.wisc.edu/writing/Handbook/DocAPA.html.

The Chicago style manual : http://www.chicagomanualofstyle.org. (users must register to use The Chicago Manual of Style search feature.)

http://www.libs.uga.edu/ref/chicago.html (Quick guide for Chicago based on 15th edition, 2003)


Use of the Web

On the Internet, search engines allow you to search broadly through databases of Web sites. This is a very powerful tool, but there are drawbacks.


Some general search engines are:

Some meta search engines (which search the search engines) are:


Some search engines specialize in certain topics. These specialized engines are useful for narrowing your search results when you are looking for a very specific type of information. Examples of specialized search engines are:

CollegeNet (http://www.collegenet.com/)

Career Builder (http://www.careerbuilder.com/)

Ecola's 24-hour Newstand (http://www.ecola.com)

Zip2 Yellow Pages (http://www.zip2.com/)


Tip

When you find research materials on the internet, record what, where, and when you found them. Record or print a complete citation for each source you find, because you may need it again later. In addition to the basic information, on-line source citations should also include:

  1. The publication medium (i.e., CD ROM or on-line)
  2. For the Internet, the URL of the site, the date when it was last updated, the name of the article and the author, the name of the organization who posted the site, and the title of the site.
  3. The date you accessed the information.
  4. You may also want to create a short summary of the site content.


Helpful Sites

The following sites are useful to learn more about search engines:

http://its.psu.edu/training/wbt/index.html
(A web-based tutorial; an introduction to the Internet and Web)

http://www2.widener.edu/Wolfgram-Memorial-Library/pyramid.htm*
(Includes search engine comparisons and search exercises)

http://www.noodletools.com/debbie/literacies/information/5locate/adviceengine.html
(Includes categories of search engines by different information needs)

http://home.sprintmail.com/~debflanagan/main.html
(Includes search strategies, search engines, and search exercises)


Heather: I usually get more information than I need when searching the Web. And, I often have to work through a number of sites before I find one that I would want to use for a research paper.

Sage: We should probably keep that in mind when searching the library for materials, too. There is a way around the problem of broad searches though.

Brian: Really?

Sage: Yes, really. Some search engines focus on particular topics instead of the entire Web.

Jose: That must be useful if you are looking for a specific type of information, like something in medicine, or education, or career information.

Sage: Exactly.

Heather: So where do you think we should begin?

Sage: Since we have a computer here let's connect to the library database (http://cat.libraries.psu.edu/). We can begin our search there.

Deena: OK. I've opened a browser. Do we want to start with a standard search?  Since we are looking up information on first-year experiences, I could try looking up something like, "college students" for the topic in the on-line library catalog.

Brian: From what we were talking about before I thought keyword searches were a better way to start. Don't we need to define our keywords first?

Jose: I think you are right, Brian. A keyword search will probably give us a better selection of information since we don't know specific titles of books or authors to look for.

Deena: OK. What are we really searching for here, experiences of first year college students?   So, what should I type to do a keyword search?

Brian: Try "freshman college experience."


Standard Search

Standard library database search strategies involve common resource identifiers (i.e., Author, Title, Subject, Heading, Keyword).

Example:

The subject term, "college students," could be used to look for information about first-year experiences.

The subject term, "pedagogy," could be used to look for information about education of children and youth.

Drawback: You must often know the exact name of the author, the exact title, or the exact subject term that the database or index uses.The terms used in different databases may vary and can affect your search results. Without exact terms, results may be unfocused and return too many records.


Keyword Search

A keyword search attempts to find key words (important, relevant, critical) not already pre-organized as identifiers (such as the subject index), but located anywhere in the entire index or database.   This type of search looks for the words in the full listing (author, subject, title, abstract, date, etc.) of each entry.  A keyword search is a less restrictive approach to finding materials containing terms you identified.   This technique can be helpful when your topic is broad, or if an index does not use identifiers which match your topic.

Example:

The keyword "galaxy" would retrieve entries about the solar system, the Milky Way, etc.

Drawback:   This form of search can be inaccurate.   A keyword may appear in many different entries--therefore many of the entries returned may have nothing to do with your topic.   Generally, you need to figure out a way to focus keyword searches to narrow down the results.   For example, the keyword, "train" could bring back entries involving locomotives.   But it may also bring back entries involving workplace training, classroom training, puppy training, weight training, and other entries.


Boolean Logic

Deena: Well, you saw what I got. 250 entries. And a lot of these don't look like what we want.

Brian: I guess we need to add more keywords or something.

Jose: What should we do now, Sage?

Sage: Let's try using Boolean logic. I learned that from reading the help on the Web search engines. It also works for many library search engines. Basically it is a way of combining word searches to help focus the results on a specific topic while still searching broadly. You can combine keywords using"and," "not," and "or."

Deena: Ok, so when we entered the keyword "experience," the database returned everything that had experience in it, right?

Sage: Right.

Deena: So to narrow things down I should do a Boolean search.

Sage: Type in the keyword, "experience," and then the word "and," and now the keyword, "freshman."

Sage: It should read as follows: "experience" and "freshman."


Note: Different search engines may use different Boolean words or symbols. Generally, you can find the correct syntax for the search engine by using the help option.

Boolean--or a Combining--search strategy can be a powerful way to overcome the weaknesses of both standard and keyword searches. This strategy allows you to use operators such as "and," "not," and "or" to search for more than one term at a time, and combine the results. This kind of strategy allows you to search broadly, while at the same time focusing your results on your specific topic.

Examples (see search engine results below):

Drawback: Boolean searching is powerful, but somewhat more complex to understand and use.

Factoid: Logician George Boole (1815 - 1864) pioneered the study of symbolic logic or mathematical logic.   This area of study examines relationships of sets of concepts and has a strong link to retrieving information.

Activity 3:

Identify the Boolean term used in each of the following searches:

  1. fish or cows
  2. fish not tropical
  3. fish and tropical

 

In addition to simple searches, search statements can be combined to be even more restrictive.    Here are some examples and the results:

fish and tropical

 

fish not tropical

 

(Whales or dolphins) and mammals

 

algebra and calculus

 

algebra and calculus not linear

Activity 4:

Check to see what you have learned about searching by taking this quiz (ANGEL Quiz) .


Evaluation Refined

Brian: All right, I was just checking my schedule and I need to get going. So, are we about finished?

Sage: Yeah, I think we've just about covered everything.

Brian: Great! Then I'm leaving.

Deena: Not so fast, Brian. I want to make sure we're all going to use what we've talked about.

Deena: I think we can refine those preliminary evaluation processes a little. You remember, currency, relevancy, clarity, and authority.

Jose: Yeah, I remember those.



Once you have done a preliminary search, how do you decide what sources, references, citations, etc. to use? You should examine carefully the content of your sources. If your source is a book, definitely read the preface and introduction in order to analyze the intentions of the author or authors. Check out the index and table of contents to get a sense or overview of what the source covers. Determine if there are chapters that speak directly to your research interest. If the source is a journal, read the abstract or editor's comments.

Remember the categories of Currency, Relevancy, Clarity, and Authority from your preliminary evaluation process? Refining that process leads to a more effective evaluation of potential sources for use in your research. You should develop your own guidelines that work for you as you proceed through higher education, but here are some additional questions that will help you practice evaluating your sources.

Currency


Relevancy


Clarity


Authority

Activity 5:

Take this quiz (ANGEL Quiz) to see if you understand how to evaluate materials.


Jose: So, what have we learned?

Brian: I think the most important thing is that if we don't use effective search strategies, we will waste a lot of time sorting through useless information.

Jose: I agree. So, we need to use advanced search strategies.

Heather: Like defining and refining exactly what we're looking for.

Sage: And using Boolean logic to narrow the kinds of information we get from a search.

Deena: I think we should remember to utilize help--whether it's a human being in the library or the help function in a search engine.

Brian: Good point. Once we have all this information, let's remember to write down the--what was that called?

Sage: Citation.

Brian: That's right, citation.

Sage: That's so we know where we found the information.

Jose: Great! It seems like we've got a pretty good handle on what we have to do.

Sage: I'd like to mention one thing that I think is really important.

Brian: What's that, Sage?

Sage: Well, I think that having the Internet adds more complexity to source evaluation--which is an already complex process. I mean, anyone can publish on the Web without certification of facts or sources. So, we need to be cautious when we're looking at information on a Web page.

Heather: Good point, Sage.


Evaluation Refined - The Tangled Web

The Internet adds complexity to source evaluation--an already complex process. The following guidelines for evaluating Web page materials are adapted from Jan Alexander and Marsha Ann Tate, Wolfgram Memorial Library, Widener University, Chester, PA. Alexander and Tate take five traditional criteria (Authority, Accuracy, Objectivity, Currency and Coverage) and link them to evaluating Web pages. These criteria are very much like the Currency - Relevancy - Authority - Clarity model described previously in this module.

Remember, anyone can publish on the Web without verification of facts or sources.   Authorship is often unknown, qualifications can be suspect, and responsibility difficult to determine. Goals and purposes of Web pages are also subject to scrutiny as they can act as a public soapbox for disseminating personal or institutional beliefs, opinions, and biases. Alexander and Tate suggest you should first identify the kind of Web page you have found.   There are five types of pages you may be considering as sources for your research:

Advocacy Web Pages

An Advocacy Web page is one sponsored by an organization attempting to influence public opinion (that is, one trying to sell ideas). The URL address of the page frequently ends in .org (organization).

Examples: National Abortion and Reproductive Rights Action League, The National Right to Life Committee, the Democratic Party, and the Republican Party.

Business/Marketing Web Pages

A Business/Marketing Web Page is one sponsored by a commercial enterprise (usually it is a page that promotes or sells products). The URL address of the page frequently ends in .com (commercial).

Examples: Adobe Systems, Inc., the Coca Cola Company, and numerous other large and small companies using the Web for business purposes.

News Web Pages

A News Web Page has a primary purpose of providing extremely current information. The URL address of the page usually ends in .com (commercial).

Examples: USA Today , Philadelphia Inquirer , CNN.

Informational Web Pages

An Informational Web page is one that purposes to present factual information. The URL   address frequently ends in .edu or .gov, as many of these pages are sponsored by educational institutions or government agencies. Examples: Dictionaries, thesauri, directories, transportation schedules, calendars of events, statistical data, and other factual information such as reports, presentations of research, or information about a topic.

Personal Home Pages

A Personal home page is a Web page that is published by an individual who may or may not be affiliated with a larger institution. The URL address of the page may have a variety of endings (including .com, .edu, etc.).

Questions to Ask About Web Pages

Criterion #1: AUTHORITY

  1. Is it clear what organization, company, or individual is sponsoring or responsible for the page?
  2. Is there a link to a page describing the goals or purpose of the organization? If it is a company, is there a link to a page where you can find out the nature of the company, who owns it, or what its products are?
  3. Is there a way of verifying legitimacy? That is, is there a phone number or postal address to contact for more information? (An email address is not enough). Is there a way of verifying the legitimacy of an individual? (Because it is difficult to verify the legitimacy of an individual, personal home pages may be a useful source for personal opinion but use extreme caution when using them as a source for factual information).
  4. Is there a statement that the content of the page has the official approval of the company or organization?
  5. If it is an organization, is it clear whether this is a page from the national or local chapter of the organization?
  6. If it is a company, is there a way to determine the stability of the company?
  7. Is there a statement giving the organization or company name as copyright holder?
  8. For news pages:
    1. Is there a non-Web equivalent version of this material that would provide a way of verifying its legitimacy?
    2. If the page contains an individual article, do you know who wrote the article and what his or her qualifications are for writing on this topic?
    3. Is it clear who is ultimately responsible for the content of the material?
  9. For informational and individual pages:
    1. Is it clear who wrote the material and are the author's qualifications for writing on this topic clearly stated?
    2. If the material is protected by copyright, is the name of the copyright holder given?


Criterion #2: ACCURACY

  1. Are the sources for factual information clearly listed so they can be verified in another source?   (If not, the page may still be useful to you as an example of the ideas of the organization, company, etc. but it is not useful as a source of factual information).
  2. Is the information free of grammatical, spelling, and other typographical errors? (These kinds of errors not only indicate a lack of quality control, but also can produce inaccurate information).
  3. If the page belongs to a company, does the page provide a link to outside sources such as product reviews or reports filed with the Securities and Exchange Commission that can be used to verify company claims?
  4. For news pages, are there editors monitoring the accuracy of the information being published?
  5. For informational pages: is it clear who has the ultimate responsibility for the accuracy of the content of the material?
  6. If there are charts or graphs containing statistical data, are the charts or graphs clearly labeled and easy to read?

Criterion #3: OBJECTIVITY
  1. Are the biases of the organization or individual clearly stated?
  2. If there is any advertising on the page, is it separate from the informational or opinion content?
  3. Is the company's motivation for the information clear?
  4. For news pages, are the editorials and opinion pieces clearly labeled?
  5. For informational pages: is the information provided as a public service?
  6. Is the information free of advertising?

 

Criterion #4: CURRENCY

  1. Are there dates on the page to indicate:
    1. When the page was written?
    2. When the page was first placed on the Web?
    3. When the page was last revised?
  2. Are there any other indications that the material is kept current?
  3. For company financial information, is there an indication it was filed with the Securities and Exchange Commission and is the filing date listed?
  4. For material from a company's annual report, is the date of the report listed?
  5. For news pages:
    1. If a newspaper, does it indicate to which edition of the paper the page belongs?
    2. If a broadcast, does it indicate the date and time the information on the page was originally broadcast?
  6. For informational pages:
    1. If material is presented in graphs or charts, is it clearly stated when the data was gathered?
  7. If the information is published in different editions, is it clearly labeled what edition the page is from?


Criterion #5: COVERAGE

  1. Is there an indication that the page has been completed, and is not still under construction?
  2. Is it clear what topics the page intends to address?
  3. Does the page succeed in addressing these topics, or has something significant been left out?
  4. Is the point of view of the organization or company presented in a clear manner with well supported arguments?
  5. For a company:
    1. Does the page include an adequately detailed description of any product?
    2. Are all of the company's products described with an adequate level of detail?
    3. Is the same level of information provided for all sections or divisions of the company?
  6. For news pages:
    1. Is there a link to an informational page that describes the coverage of the source?
    2. If you are evaluating a newspaper page and there is a print equivalent, is there an indication of whether the Web coverage is more or less extensive than the print version?
  7. For informational pages:
    1. If there is a print equivalent to the Web page, is there a clear indication of whether the entire work is available on the Web?
  8. If the material is from a work with an expired copyright, has there been an effort to update the material to make it more current?

Useful resource: http://www3.widener.edu/Academics/Libraries/Wolfgram_Memorial_Library/Evaluate_Web_Pages/659/

Activity 6:

Now take some time to practice evaluating sources. Print the worksheet in the file below and follow the directions. The worksheet is available in four different file formats to accomodate different computer configurations. Please choose the file format most likely to work with your computer system.

Word file
RTF file

Activity 7: Internet Hunt

By now, you've probably had several opportunities to work with Internet search engines. But, have you decided which search engine you prefer? Do you know why? Are you able to find information on the Internet easily?

Make up four questions to use for a virtual scavenger hunt. These can be any type of questions--creating useful ones you will be able to apply to your studies is a good idea. The questions could even be about how Internet Search Engines work.

Use at least two search engines to find answers to your questions. Keep track of how you found your information, how long it took, what kind of information you retrieved, how many "hits" there were, when you did your search, which web site visited had the best answer, what problems you encountered or surprises you uncovered, what was the most direct route to the answer, etc.

How did your searching go? Answer the following questions using the "My Notes" tool on the right side of the tool bar at the top of the page.


Brian: Now are we done? I really need to go.

Heather: Yes, I think we've covered everything we need to make sure we find credible sources of information for our research projects.


Return to the Source Evaluation Lesson page by closing this page.

| iStudy | Module Descriptions | Getting Access | Contacts  | FAQ