back to fall 2003 • fall
2003 readings
fall 2003 web forum readings
Here are some excerpts to help with our discussion. It is not necessary
to read these articles to join the discussion. This list is by no means complete
and we welcome any suggestions you have for other readings.
The first two are from Focus on Basics and are about qualitative and quantitative
research. They give us a little background on the difference.
The third piece is a description of the IALS from the IALS web page.
There are two pieces by Mary Hamilton.
The first appears in the journal and in it she discusses which literacies
are included in the IALS data and which are missed. In the second excerpt,
she describes the ‘vernacular’ literacies she found missing from the
IALS data.
We have included three excerpts from Thomas Sticht.
The first is the one that is printed in the journal and looks at the difference
between self-assessment and performance assessment in the IALS.
The second addresses the point that Susan Sussman makes in the interview “that the IALS test requires that people have an 80 per cent probability
of responding correctly to questions at a given literacy level, in order
to count as fitting into that literacy level. One of the designers of the
test, according to Sussman, has said that a more realistic “pass” rate
would be 50 per cent.”
In the third, he discusses how the IALS data can be used to refute “the
concerns for global competitiveness that some policy-oriented reports
have used to focus adult literacy education on workforce development.”
The last three pieces are excerpt from the debate between Stan Jones and Brian
Street about whether or not the IALS privileges certain literacies
and if it does, what that means to literacy learners and workers.
- from Research with Words: Qualitative
Inquiry
by Glynda Hull
FOCUS ON BASICS.
Vol 1, Issue A • Feb 97
Full text available at http://gseweb.harvard.edu/~ncsall/fob/1997/hull.htm
- from Understanding Quantitative
Research about Adult Literacy
by Thomas Valentine
FOCUS ON BASICS,
Vol 1, Issue A • Feb 97
Full text available at http://gseweb.harvard.edu/~ncsall/fob/1997/valen.htm
- from Reading the Future: A Portrait
of Literacy In Canada
Backgrounder
Full text available at http://www.nald.ca/NLS/ials/ialsreps/ialsbk1.htm
- from Privileged Literacies:
Policy, Institutional Process and the Life of the IALS
by Mary Hamilton, Language and Education, 2001, 15 (2,
3): 178-196
- from Becoming expert: using ethnographies of everyday learning
to inform the education of adults
Papers from the 28th Annual SCUTREA Conference
Research, Teaching
and Learning: making connections in the education of adults
by Mary Hamilton,
Literacy Research Group, Department of Educational Research, Lancaster
University, UK.
http://www.leeds.ac.uk/educol/documents/000000719.htm
- from The International Adult
Literacy Survey: How well does it represent the literacy abilities of adults?
by Thomas G. Sticht, The Canadian Journal for the Study of Adult Education,
Vol, 15, No. 2 November, 2001, 15(2): pp19-36
- from How Many Low Literate
Adults Are There in Canada, the United States, and United Kingdom? Should
the IALS
Estimates be Revised?
Research Note #1 March 5, 1999 Applied
Behavioral & Cognitive Sciences,
Inc.
by Thomas G. Sticht
Full text available at http://www.nald.ca/fulltext/sticht/resnote.htm
- from From International Competitiveness
to International Inequality: New Perspectives on Social Justice From the
International Adult Literacy Survey (IALS)
Research Note 12/13/00
by Thomas G. Sticht
International Consultant in Adult Education
Full text available at http://www.nald.ca/fulltext/sticht/dec00/cover.htm
- from Ending the myth of the
'Literacy Myth'
by Stan Jones
Center for the Study of Adult Literacy, Carleton University
Full text available
at http://www.nald.ca/province/que/litcent/Publication_Products/working/page17.htm
- from Literacy, Economy and
Society: A Review
by Brian V. Street, King's College, University of London
http://www.nald.ca/province/que/litcent/Publication_Products/working/page7.htm
- from Ending the myth of the
'Literacy Myth'
by Stan Jones
Center for the Study of Adult Literacy, Carleton University
Full text available
at http://www.nald.ca/province/que/litcent/Publication_Products/working/page17.htm
Back to top
1) from Research with Words: Qualitative Inquiry
Most generally speaking, the purpose of qualitative research is to understand
human experience to reveal both the processes by which people construct meaning
about their worlds and to report what those meanings are. But what particular
kinds of information can qualitative studies offer literacy specialists and
adult educators?
Such research can reveal how people experience educational activities like
literacy classes or work related education programs -- what they value, what
they reject, what they learn, how they change. Thereby the studies can tell
us something about how and why such programs succeed and fail. This kind
of research can also document and characterize the diversity and complexity
of literacy activities as they occur in school, work, and daily life, as
well as the incentives and disincentives that people perceive for developing
and exercising literacy abilities. Thereby we can more fully appreciate the
nature of the literacy practices we are attempting to teach. And such studies
can introduce us to situations from the points of view of varied participants,
bringing to the fore individual perspectives, histories, and proclivities,
as well as structures of power that influence what people learn and are able
to do. Thereby we can place literacy learning properly in broader historical,
sociocultural, and political milieus, learning how learning is influenced
by forces outside the classroom.
by Glynda Hull
FOCUS ON BASICS,
Vol 1, Issue A • Feb 97
Full text available at http://gseweb.harvard.edu/~ncsall/fob/1997/hull.htm
Back to top
2) from
Understanding Quantitative Research about Adult Literacy
Statistical research is not as formidable as it appears, but it requires
a special type of reasoning. Statistical reasoning involves a tight, detailed,
and codified logic that can be especially difficult for people who would
rather deal in broad strokes and big ideas than with the making of fine distinctions
about extremely well focused concepts.
Some people view statistics with a sense
of moral indignation at the fact that statistics reduces things of
human importance to numbers, and they relate statistics to the power
that statistics could give to a "big brother" type
of government or to a scorn of bean-counting bureaucrats. In reality,
of course, statistical research reduces an object of study no more
than a camera reduces the object of a photograph. Statistical reasoning
simply represents a highly patterned and highly public way of looking
at the world, and, because its details can be readily scrutinized and
evaluated, it is often preferred by funding agencies and program evaluators
over more subjective and less public ways of reasoning. Like all research
methods, it can be used for good or bad purposes.
by Thomas Valentine
FOCUS ON BASICS,
Vol 1, Issue A • Feb 97
Full text available at http://gseweb.harvard.edu/~ncsall/fob/1997/valen.htm
Back to top
3) from Reading the Future: A Portrait of Literacy In Canada
Backgrounder
The International Adult Literacy Survey (IALS) is part of a continuing tradition
of attempts to measure literacy levels in the adult population by means of
surveys and to produce international comparisons. Such research is driven
by the search for universals in the relationships between literacy, education
and prosperity, which can be used to further the goal of global development.
...
Measuring literacy: More than one gauge
Literacy cannot be narrowly defined as a single skill that enables people
to deal with all types of text. People in industrialized countries face many
different kinds of written material every day, and they require different
skills to understand and use the information. To reflect this complexity,
IALS developed three categories of literacy:
1 . Prose literacy: the ability to understand and use information from texts
such as editorials, news stories, poems and fiction.
2. Document literacy: the ability to locate and use information from documents
such as job applications, payroll forms, transportation schedules, maps,
tables and graphs.
3. Quantitative literacy: the ability to perform arithmetic functions such
as balancing a chequebook, calculating a tip, or completing an order form.
...
A broad perspective
If economies require increasing numbers of highly skilled workers to expand,
then growth will be affected by existing practices of employers, 'individuals
and governments: IALS has shown that instead of enlarging the pool of highly
skilled workers, the tendency is to increase the skills of the already skilled.
The reserve employment pool, made up of the unemployed and those working
in declining industrial sectors, is low-skilled. Policies directed towards
providing more educational opportunities and increasing skills in that pool
must he a necessary part of any industrial growth strategy.
The distribution of literacy is also a good predictor of the magnitude of
differences between social groups, making literacy an essential element for
promoting social cohesion. Therefore, any view of literacy which is focussed
on economic objectives alone is untenable.
Full text available at http://www.nald.ca/NLS/ials/ialsreps/ialsbk1.htm
Back to top
4) from Privileged Literacies: Policy, Institutional Process and the Life
of the IALS
The IALS draws on a particular discipline – the psychometric measurement
tradition. It uses an information processing model of literacy and attempts
to identify levels of literacy skill that are independent of the context
of use – the literacy counterpart of the generic and transferable labour
skills supposedly possessed by the flexible worker (cf also Bernstein’s
(1996: 56) new performance pedagogies). It creates three dimensions of
literacy: prose, document and quantitative. In generating test items,
the starting points were texts taken from real-life contexts in a range
of countries especially NorthAmerica. They include bus timetables, advertisements
and consumer instructions. These then underwent various transformations
to turn them into test items and those showing cultural bias or linguistic
translation problems were dropped. The final test rests on 35 texts,
each one used as the basis for several question items. The tests are
designed to ensure a broad spread of responses across an arbitrarily
fixed set of five levels.
...
The test items cover commercial, financial,
media, advertising and entertainment related texts; work-place, including
job-seeking; consumer manuals and instructions; transportation-related
and a recipe. In general, the test items require reading and formulaic
writing responses (such as form-filling) rather than compositional
writing which is difficult to subject to standardised scoring. Not
included in the test are religious texts, letters (personal or official),
greetings/condolences; legal documents, political, government and
policy documents; literature (novels, drama or poetry); historical
writing; autobiography; humour and satire. This is, therefore, a
very limited, standardised ‘generic’ view of
literacy presented as a universal standard and from which culturally
specific material has been partialled out.
Back to top
5) from Becoming expert: using ethnographies of everyday learning to inform
the education of adults
In our project we found vernacular literacies involved in a range of everyday
activities, which we roughly classified as (1) organising life (2) personal
communication (3) private leisure (4) documenting life (5) sense making and
(6) social participation.
...
Firstly, vernacular literacy practices are learned informally. They are
acquired in homes and neighbourhood groups, through the everyday perplexities
and curiosities of our lives. The roles of novice or learner and expert or
teacher are not fixed, but shift from context to context and there is an
acceptance that people will engage in vernacular literacies in different
ways, sometimes supporting, sometimes requiring support from others.
Secondly, the vernacular literacy practices we identified are rooted in
action contexts and everyday purposes and networks. They draw upon and contribute
to vernacular knowledge, which is often local, procedural and minutely detailed.
Literacy learning and use are integrated in everyday activities and the literacy
elements are an implicit part of the activity. which may be mastering a martial
art, paying the bills, or finding out about local news. Literacy itself is
not a focus of attention, but is used to get other things done.
by Mary Hamilton, Literacy Research Group, Department of Educational Research,
Lancaster University, UK.
Back to top
6) IALS Methodology and Validity
from The International Adult Literacy Survey: How well does it represent
the literacy abilities of adults?
In the IALS, the performance scales and
the self-assessments represent two fundamentally different approaches
to assessing literacy abilities. In the performance assessments,
literacy is construed as a cognitive ability (latent trait) that
makes possible the use of printed materials in various contexts.
It is considered that some people have more of this capacity than
others, although how much people have or lack may not be consciously
apparent to them. Nonetheless, it is assumed that these differences
in the amount of capacity can be inferred using people’s performance on various real-world
tasks that incorporate the latent trait that is theorized to make possible
each person’s performance.
In the self-assessment approach to assessing literacy, literacy is considered
as an ability or set of abilities (as in reading, writing, and numeracy in
the IALS) that adults are consciously aware of and can perceive well enough
to estimate how well their literacy skills permit them to negotiate the literacy
demands of different sets of activities at work or in their daily life. This
requires that adults are aware both of the demands for literacy in the different
contexts that they encounter and of how well their literacy abilities permit
them to meet these demands on a recurrent basis.
Clearly, these two different approaches to assessing literacy are based
on different implicit theories about literacy and different procedures for
measuring literacy. It is also evident from the discrepancies in data that
these approaches produce different estimates of how many adults are at risk
because of literacy in the various nations that participated in the IALS.
These findings raise serious questions about the validity of the different
assessments. Is each assessment equally valid as a means of representing
the literacy abilities of the adult population? If so, then how should the
different results of each method be used?
by Thomas G. Sticht, The Canadian Journal for the Study of Adult Education,
Vol, 15, No. 2 November, 2001, 15(2): pp19-36
Back to top
7) from How Many Low Literate Adults Are There in Canada, the United States,
and United Kingdom? Should the IALS Estimates be Revised?
Research Note #1 March 5, 1999 Applied Behavioral & Cognitive Sciences,
Inc.
Issues in Setting Standards For Adult Literacy.
When the National
Adult Literacy Survey (NALS) was released in 1993 I raised the question
as to why it was that 66 to 75 percent of adults in Literacy Level
1, the lowest level, thought they could read and write "well" or "very well," figures
that were similar to those found in the later IALS for the three nations
under discussion. I noted that this might result from the fact that
on the Prose scale of the NALS, people with a score of 200 were assigned
to Level 1 because they had an 80 percent probability of being able
to do the average task at level 1. However, these same people would
be expected to be able to respond correctly to over 45 percent of
the average tasks at Level 2, 25 percent of the average tasks at
Level 3, and even 15 percent of the average tasks at Level 5, the
highest level of literacy. Yet, because they were assigned to Level
1, all competence above that level was denied to them. Similar findings
held for Document and Quantitative scales.
...
Both the International Adult Literacy Survey (IALS) and the National Adult
Literacy Survey (NALS) set a criterion of having an 80 percent probability
of getting the average item at a given Literacy Level correct to be assigned
to that level of skill. This resulted in some 20.7 percent of adults in the
United States being classified as in the lowest level of literacy.
However, Andrew Kolstad, the leader of the NALS project at the National
Center for Education statistics has argued that the 80 percent probability
level is arbitrary (Kolstad, 1996). He calculated the percentage of adults
who would be in Level 1 if a criterion of 65 percent, which is used by the
National Assessment for Educational Progress (NAEP) for the K-12 system,
was used. In this case the percentage of adults scoring below Level 3 dropped
to 32, a reduction of some 15 percent.
Kolstad also determined the consequences of using the 50 percent probability
criterion used by the widely employed Comprehensive Adult Student Assessment
System (CASAS) which is included in the federal government's dissemination
network. In this case, only 9 percent were in Level 1, the lowest level of
literacy.
...
In more recent analyses, Kolstad (personal
communication, January 26, 1999) has demonstrated that the use of
the 50 percent probability criterion produces the least errors in
predicting whether adults can or cannot perform literacy tasks across
the full range of tasks included in the NALS. At the present time
in the United States, stimulated by Kolstad’s work, there is considerable
debate going on at the National Center for Education Statistics about
just which standards should be used for all national assessments, those
for school children and adults as well (Kolstad, et al., 1998).
by Thomas G. Sticht
Full text available at http://www.nald.ca/fulltext/sticht/resnote.htm
Question: How many functionally illiterate adults
are there in the United States?
Wag's Answer: As many as you would like!
from Has the National Adult Literacy Survey (NALS) Defamed the Competence
of America's Labor Force? Research Note 10 July 2001
by Thomas G. Sticht
http://www.nald.ca/fulltext/sticht/july01/cover.htm
Back to top
8) from From International Competitiveness to International Inequality:
New Perspectives on Social Justice From the International Adult Literacy
Survey (IALS) Research Note 12/13/00
... the influential Jump Start report of
1989 stated, "There is no way
in which the United States can remain competitive in a global economy, maintain
its standard of living, and shoulder the burden of the retirement of the
baby boom generation unless we mount a forceful national effort to help adults
upgrade their basic skills in the very near future (p.iii)." Yet
now we find the U.S. pre-eminent in the global economy, with very low
unemployment rates internally, and on a par with the world's leading
economic western nations in terms of average adult literacy skills. Overall
then, there is not much of a basis in the report for arguing that the
U. S. is not economically competitive internationally because of low
adult literacy. Hence such economically-based arguments are likely to
be less influential in the foreseeable future for advocating for adult
literacy education.
...
The report goes on to note that ".inequality
in the range of literacy scores in North America is also among the
highest of the nations surveyed. Especially in he United States,
inequality in the distribution of literacy scores on the English
test [that is, the NALS] used for the survey is strongly related
to economic inequality measured by income differentials between households."
It seems to me, then, that the emphasis of this recent report using IALS
data is largely on the inequality of literacy among adults within nations,
and the economic consequences of these differences in literacy for adults
within a given nation. In many respects, this seems to be somewhat of a change
in perspective from the concern for adult literacy as a factor in international
competitiveness that has in large part driven the Workforce Investment Act
of 1998, to a return to the concern for issues of poverty and the need for
individuals to be economically competitive within our nation that led to
the enactment of the adult basic education program as part of the War on
Poverty's Economic Opportunity Act of 1964. In a sense, with this new report
using IALS data, we seem to have gone back from the concerns with international
competitiveness of A Nation at Risk of the 1980s and 1990s to the concerns
for People at Risk of the 1960s. This might be a more fruitful stance for
advocating for the full recognition of the Adult Education and Literacy System
(AELS) as the third major, mainstream component of our nation's publicly
supported educational structure (K-12, AELS, Higher Education,) for promoting
the general health, welfare and prosperity of the nation. It might also augur
well for placing workforce development in a more appropriate, tertiary position
with regard to its importance as an outcome for adult education and for getting
the WIA changed to the Adult Education, Literacy and Workforce Investment
Act (AELWIA) when it next comes up for reconsideration.
by Thomas G. Sticht
International Consultant in Adult Education
Full text available at http://www.nald.ca/fulltext/sticht/dec00/cover.htm
Back to top
9) from Ending the myth of the 'Literacy Myth'
One of the strongest claims in IALS is that there is an important relationship
between literacy skill and individual economic success.
The evidence for this lies in a number of observations in the study:
- There is a relationship between income and literacy skill.
For example, in Canada IALS found that over 80% of those at the lowest literacy
level had incomes below the median quintile, but just 42% of those at the
highest level had incomes this low. Over a quarter of those at the highest
level had incomes in the top quintile, but only 5% of those at the lowest
level had incomes this high (p. 61 of the report).
- There is a relationship between labour force attachment and
literacy.
For example, in Sweden 23% of those at the lowest literacy level were unemployed
vs. just 2% among those at the highest level (p.58).
- There is a relationship between occupational change and literacy.
For example, in Germany the average score of those working in industries
showing the greatest growth was significantly higher than the average score
of those working in essentially stagnant industries (p. 65).
...
Literacy ability is a factor in work force participation, it is a factor
in social participation. IALS shows this to be the case.
In many ways the IALS message is not new to the literacy field. Anyone who
has worked with learners understands that this message is not new to them.
It does appear to be new to the people who decide whether there should be
funding for literacy programs. The message from the literacy research community
in Canada that literacy didn't matter, the same message Street and Graff
want us to believe, had nearly succeeded in convincing governments that investments
in adult literacy were not worthwhile.
by Stan Jones
Center for the Study of Adult Literacy, Carleton University
http://www.nald.ca/province/que/litcent/Publication_Products/working/page17.htm
Back to top
10) from Literacy, Economy and Society: A Review
The people at lower levels, however, report on themselves as having sufficient
literacy skills for their purposes - they do not rate themselves as 'poor'
and indeed many rate themselves as 'excellent' (p. 109). This suggests that
they apprehend Scribner and Cole's insight that it would be inappropriate
to apply the literacy standards of one domain or cultural to those required
in another: literacy is specific to context.
The researchers themselves certainly pay lip service to this position and
have done a service in moving beyond the dominant view in agency and government
surveys that there is a single standard to be applied to all, and that there
are just two dimensions in measurement - literacy and illiteracy. They rightly
reject the latter term as unhelpful in contemporary society, with its unfortunate
connotations of ignorance and deficit. In practice, levels and demands on
literacy are varied and are changing all the time, so the researchers instead
are attempting to design scales that will capture this complexity and variation.
Instead of a simple literacy/illiteracy dichotomy, they measure literacy
in three 'domains' - prose literacy, document literacy and quantitative literacy
- across which individuals will vary according to experience and context.
However, there are indications throughout the report that the authors, and
certainly those who make use of the findings, will in practice resist this
call to complexity, and continue to privilege certain kinds of literacy and
certain types of knowledge as superior to others. Indeed, that in a sense
is the motivation for producing the report in the first place: to tell governments
and agencies where their populations are failing so that they can put it
right. The language of deficit runs through the report of findings and the
commentaries on it.
...
But even before we address these issues, it is significant that the report
has reduced the discussion of literacy to these work-oriented issues and
to a traditional 'functional' approach to literacy.
The authors' definition of literacy itself reproduces the standard functional
view of early UNESCO documents: 'Using printed and written information to
function in society: to achieve one's goals, and to develop one's knowledge
and potential' (p.14).
This view of literacy has been elaborated, refined and frequently rejected
by more sophisticated recent research. But none of this is mentioned in the
report, which proceeds as though the definitions and the terms are unproblematic
and universally agreed. Similarly the language of deficit already hinted
at in the opening reference to 'inadequate levels of literacy' recurs throughout
as a leitmotif with constant suggestions of a literacy 'problem,' of 'low'
levels, of 'success' or 'failure' in achieving on the tests and of 'remediation'
in putting it right.
...
If we instead pose the question of whose
interpretations are 'appropriate,' applying a key concept from the
Ethnography of Communication tradition, the frame shifts from a monolithic
quasi-scientific assumption of truth to a more socially relative
recognition that particular discourses and utterances are appropriate
to particular social conditions: Scribner and Cole's "specific
practices promote specific skills."
If the IALS researchers wish to argue that the conditions set by their survey,
and in particular by the test items and the questions respondents were asked
about them, lead to certain kinds of response as more 'appropriate,' then
they need to characterize that domain more carefully. They need to problematize
its assumptions rather than to take them for granted and to indicate something
of how other responses would be appropriate in other conditions.
There is a power relation, then, between the researchers and their respondents,
on the one hand, and between this particular style of research and other
research traditions, on the other. The research team indeed have immense
power as the very debate now going on about their findings indicates.
That they do not draw attention to this power but instead write as though
their findings are the neutral product of objective scientific inquiry is
itself a classic procedure of institutional power. If nothing else, the report
will provide excellent data for students interested in the workings of discursive
power in late twentieth century Europe and N. America.
by Brian V. Street, King's College, University of London
http://www.nald.ca/province/que/litcent/Publication_Products/working/page7.htm
Back to top
11) from Ending the myth of the 'Literacy Myth'
The specific evidence from the statistical analysis of IALS is that there
is simply no general evidence of a situation effect. If previous experience
were all that counted, as Street seems to say, then we would expect someone
who answered one of the questions based on the bus schedule correctly to
answer the rest of the questions on the bus schedule correctly. This doesn't
happen. Instead. the best predictor of whether someone can answer a particular
question on the bus schedule correctly is how well they answer questions
with similar cognitive demands on other, situationally unrelated texts.
But let's assume that Street and Graff are right that literacy is narrowly
situationally specific. Then it would seem that IALS has managed to locate
and test a situationally specific literacy that is highly related to social
and economic well-being.
Street and Graff are sure to claim that we have thus privileged this one
kind of literacy. But it is not the IALS researchers who have privileged
it, it is society. While we might determine test scores, we don't determine
employment, income, social participation or any of the other characteristics
we found associated with IALS literacy. It is not for the IALS research team
to determine whether it is fair that this one kind of literacy is so valued
by society. It would have been negligent of us, however, having discovered
these connections not to have reported them.
by Stan Jones
Center for the Study of Adult Literacy, Carleton University
http://www.nald.ca/province/que/litcent/Publication_Products/working/page17.htm
