WordStat 5.1 Computer Assisted Text Analysis

Hiller's Communication Vagueness Scale

Vagueness is a psychological construct which refers to the 'state of mind' of a communicator who does not sufficiently command the facts, knowledge, or understanding required for maximally effective communication. Vagueness is an internal stimulus condition principally determined by a performer's instant command of knowledge, by his motivation to communicate, and by his own evaluation of imperfect success. Use of the words and phrases in the Vagueness dictionary provide the analyst with a clue to the communicator's vague state of cognition. to uncertainty of facts and/or understanding of the thought underlying the communication. So, for example, when a speaker hesitates and then says, Well, as "you all know," and "you know," or says, "clearly," "obviously," or "naturally," the analyst may infer that most folks in the audience do not know, and the reasons are not clear to the speaker, so that the speaker has opted to bluff his way through the topic to be addressed.

Ten separate categories of vagueness have been defined to permit an optional finer analysis of the nature of the vagueness exhibited (see table below).

SCALES	DESCRIPTION	EXAMPLES	# ITEMS
Ambiguous Designation	Something potentially specifiable is referred but not definitely identified.	somewhere, stuff, a certain, and so on	50
Negated Intensifiers	Negations can be evasions.	not always, not quite, isn't necessarily	56
Approximation	Use reflects real or referential vagueness or imprecise knowledge.	nearly, sort of, fairly, almost, pretty much	35
Bluffing and Recovery	Used when a writer is not communicating effectively and tries to shift responsibility for making sense of content to the reader.	actually, anyway, as a matter of fact, of course	53
Admission of Error	Repeated admissions indicate lack of confidence or lack of competence.	I made a mistake, maybe, I don't know	20
Indefinite Amount	An amount is potentially knowable but is not specified.	a bit, a bunch, a couple, a little, some	29
Multiplicity	Pseudospecification or glossing over of complexity.	aspects, types, lots, factors, kinds	35
Probability and Possibility	Indicates lack of clarity or lack of definite knowledge.	at times, could be, generally, perhaps	33
Reservations	Expressions of doubt or reluctance to commit to a definite point of view.	apparently, appears, relatively, seems.	35
Anaphora	Excessive and repetitious use of pronouns rather than direct references makes content more difficult to follow.	former, she, she, it, latter, other, them	14

The scale was originally developed to analyze university lecture delivery but can also be used to detect vagueness in other sources.

WordStat format

The WordStat format contains 362 words and phrases stored in 10 categories. An analysis at the first level gives a global score on the total of those items. Setting the level of analysis to 2 provides scores for the 10 subscales.

To purchase the WordStat version of the Communication Vagueness Scale, select the proper link below:

Product	Price (US$)
Single User - Communication Vagueness Scale	$50	Buy it!
Site License - Communication Vagueness Scale	$200	Buy it!
Academic User - Communication Vagueness Scale	$25	Buy it!
Academic Site License - Communication Vagueness Scale	$100	Buy it!

Main References

Hiller, J.H., Marcotte, D., & Martin, T. (1969). Opinionation, vagueness, and specificity-distinctions: Essay traits measured by computer. American Educational Research Journal, 6(2), 271-286.

Abstract: Three characteristics of writing quality were selected for study from among the many discussed in English texts on the assumption that single words or discrete phrases reliably cue the presence of such characteristics in essays. A set of 256 graded essays was searched by computer for cues, and the measures thus obtained were correlated with the essay grades. All predicted correlations were significant at p < .01.

Hiller, J.H., Fisher, G., & Kaess, W. (1969). Computer investigation of verbal characteristics of effective classroom lecturing. American Educational Research Journal, 6(4), 661-675.

Abstract: After the research on essay grading, the Vagueness construct was applied to data collected by the Stanford Explanation Project (se Gage, N.L, Belgard, M. Dell, D., Hiller, J., Rosenshine, B., and Unrah, W, "Explorations in the Teachers Effectiveness in Explaining." Tech. Report 4 of the Stanford Center for R&D in Teaching, School of Education, Stanford University, 1968). Teachers delivered self-prepared lectures to their regular classrooms of students based on two magazine articles, one about Thailand and the other about Yugoslavia. Their students were administered standardized, objectively scored multiple choice knowledge tests after the lectures whose scores served as the criterion for teacher performance. The class test scores were refined by covarying out class performance on a test for a third lesson (on Israel) that was delivered by a single lecture recorded by a professional speaker, thus removing variance having to do with more or less able and motivated individual classrooms of students. In the available sample of 32 lectures about Yugoslavia, the correlation between proportion of Vagueness items in the lectures (vagueness items counted divided by total number of words for the given lecture) and the test scores was -,59 (p<.005), and in the available subsample for the Thailand lecture, the correlation was -. 48, N= 23, p<.05

Hiller, J.H. (1971). Verbal response indicators of conceptual vagueness. American Educational Research Journal. 8(1), 151-161.

Abstract: The plight of being caught without a needed word while speaking or writing is a common experience. A sudden lapse in memory or momentary confusion may underlie the difficulty. Or the problem may arise when a speaker, in the midst of expression, discovers he doesn't command the knowledge required to complete his gambit. This paper presents an analysis of the verbal behavior of the speaker or writer confronted with the immediate necessity to express himself while he struggles against one or more such handicaps. The key term in this analysis—vagueness—is defined here, after which evidence obtained in two correlational studies is reviewed and then recent experimental evidence reported.

Studies Using the Communication Vagueness Scale

Andre, M.E.D.A. & Anderson, T.H. (1979). Development and evaluation of a self-questioning study technique. Reading Research Quarterly, 14(4), 605-623.

Borg, W.R. (1975). Protocol materials as related to teacher performance and pupil achievement. Journal of Educational Research, 69(1), 23-30.

Carrier, C.A. & Fautschpatridge, T. (1981). Levels of Questions: A framework for the exploration of processing activities. Contemporary Educational Psychology, 6(4), 364-382.

Chaudron, C. & Richard, J.C. (1986). The effect of discourse markers on the comprehension of lectures. Applied Linguistics, 7(2), 113-127.

Chilcoat, G.W. (1987). Teacher talk: Keep it clear. Academic Therapy, 22(3), 263-271.

Civikly, J.M. (1992). Clarity: Teachers and students making sense of instruction. Communication Education, 41(2). 138-152.

Duchastel, P.C. & Nungester, R.J. (1984). Adjunct question effects with review. Contemporary Educational Psychology, 9(2), 97-103.

Duchastel, P.C. (1983). Interpreting adjunct question research: Processes and ecological validity. Human Learning, 2(1), 1-5.

Dunkin, M.J. & Doenau, S.J. (1982). Ethnicity, classroom interaction and student achievement. Australian Journal of Education, 26(2), 171-189.

Dunkin, M.J. (1978). Student characteristics, classroom processes, and student achievement. Journal of Educational Psychology, 70(6), 998-1009.

Dunking, M.J. & Doenau, S.J. (1980). A replication study of unique and joint contributions to variance in student achievement. Journal of Educational Psychology, 72(3), 394-403.

Faw, H.W. & Waller, T.G. (1976). Mathemagenic behaviors and efficiency in learning from prose materials: Review, critique and recommendations. Review of Educational Research, 46(4), 691-720.

Feldman, C.F. & Wertsch, J.V. (1976). Context dependent properties of teachers speech. Youth & Society, 7(3), 227-258.

Ford, J.M., Stetz, T.A., Bott, M.M. & O'Leary, B.S. (2000) Automated content analysis of multiple-choice test item banks. Social Science & Computer Review, 18, 258-271.

Gagne, E.D. (1978). Long term retention of information following learning from prose. Review of Educational Research, 48(4), 629-665.

Galindo, I (1998). Say what you mean and mean what you say. Church Teaching Today, 6(7).

Gliessman, D.H., Pugh, R.C., Brown, L.D., Archer, A.C., & Snyder, S.J. (1989). Applying a research based model to teacher skill training. Journal of Educational Research, 83(2), 69-81.

Glover, J.A., Zimmer, J.W., Pilbeck, R.W., & Plake, B.S. (1980). Effects of training students to identify the semantic base of prose materials. Journal of Applied Behavior Analysis, 13(4), 655-667.

Hamaker, C. (1986). The effect of adjunct questions on prose learning. Review of Educational Research, 56(2), 212-242.

Hamilton, R.J. (1985). A framework for the evaluation of the effectiveness of adjunct questions and objectives. Review of Educational Research, 55(1), 47-85.

Heath, R.W. (1974). Research basis for performance based teacher education. Review of Educational Research, 44(4), 463-484.

Janzen, H.L. & Hallworth, H.J. (1977). Linguistic predictors of properties of set. Alberta Journal of Educational Research, 23(1), 11-21.

Janzen, H.L. (1972). Investigation of integral nature of soviet concept of set. International journal of psychology, 7(4), 207-218.

Johnson, L.L. & Otto, W. (1982). Effect of Alterations in prose style on the readability of college texts. Journal of Educational Research, 75(4). 222-229.

Klare, G.R. (1976). 2nd look at validity of readability formulas. Journal of Reading Behavior, 8(2), 129-152.

Land, M.L. & Smith, L.R. (1979). Effect of a teacher clarity variable on student achievement. Journal of Educational Research, 72(4), 196-197.

Land, M.L. & Smith, L.R. (1979). Effect of low inference teacher clarity inhibitors on student achievement. Journal of Teacher Education, 30(3), 55-57.

Land, M.L. (1979). Low inference variables of teacher clarity: Effects on student concept learning. Journal of Educational Psychology, 71(6), 795-799.

Land, M.L. (1980). Teacher clarity and cognitive level of questions: effects on learning. Journal of Experimental Education, 49(1), 48-51.

Land, M.L. (1981). Combined effects of 2 teacher clarity variables on student achievement. Journal of Experimental Education, 50(1), 14-17.

Lysakowski, R.S. & Walberg, H.J. (1982). Instructional effects of cues, participation, and corrective feedback: a quantitative synthesis. American Educational Research Journal, 19(4), 559-578.

Murray, H.G. (1983). Low inference classroom teaching behaviors and student ratings of college teaching effectiveness. Journal of Educational Psychology, 75(1), 138-149.

Nuthall, G. (1972). Observation systems used with recording media. International Review of Education, 18(4), 491-507.

Petersen, C., Glover, J.A. & Ronning, R.R. (1980). Examination of 3 proses learning strategies on reading comprehension. Journal of General Psychology, 102(1), 39-52.

Rickards, J.P. (1976). Interaction of position and conceptual level of adjunct questions on immediate and delayed retention of text. Journal of Educational Psychology, 68(2), 210-217.

Rickards, J.P. (1976). Type of verbatim question interspersed in text. Journal of Reading Behavior, 8(1), 37-45.

Rickards, J.P. (1977). Inserting Questions before or after segments of text. Contemporary Educational Psychology, 2(2), 200-206.

Rickards, J.P., Anderson, M.C., & McCormick, C.B. (1976). Processing effects of common words and number questions inserted in reading materials. Journal of Educational Research, 69(7), 274-277.

Rosenthal, T.L. & Downas, A. (1985). Cognitive aids in teaching and treating. Advances in Behaviour Research and Therapy, 7(1-2), 1-53.

Schumacher, G.M., Moses, J.D., & Young, D. (1983). Students studying processes on course related texts: The impact of inserted questions. Journal of Reading Behavior, 15(2), 19-36.

Shavelson, R. & Dempsey-Atwood, N. (1976). Generalizability of measures of teaching behavior. Review of Educational Research, 46(4), 553-611.

Smith, B.O. (1985). Research bases for teacher education. Phi Delta Kappa, 66(10), 685-690.

Smith, L.R. & Bramblett, G.H. (1981). The effect of teacher vagueness terms on student performance in high school biology. Journal of Research in Science Teaching. 18(4), 353-360.

Smith, L.R. & Cotten, M.L. (1980). Effect of lesson vagueness and discontinuity on student achievement and attitudes. Journal of Educational Psychology, 72(5), 670-675.

Smith, L.R. (1984). Effect of teacher vagueness and use of lecture notes on student performance. Journal of Educational Research, 78(2), 69-74.

Smith, L.R. (1985). Presentational behaviors and student achievement in mathematics. Journal of Educational Research, 78(5), 292-298.

Smith, L.R. (1985). Teacher clarifying behaviors: Effects on student achievement and perceptions. Journal of Experimental Education, 53(3), 162-169.

Snow, R.E. (1974). Representative and quasi-representative designs for research on teaching. Review of Educational Research, 44(3), 265-291.

Weintraub, S., Robinson, H.M., Smith, H.K., Plessas, G.P., Roser, N.L. & Rowls, M. (1976). Summary of investigations relating to reading: July 1, 1974, to June 30, 1975. Reading Research Quarterly, 11(3), 223-563.

Wilson, S.M. & Wineburg, S.S. (1993). Wrinkles in time and place: Using performance assessments to understand the knowledge of history teachers. American Educational Research Journal, 30(4), 729-769.