Approaches to Cognitive Science - Course Materials: the `ATCS Book'

Preface

ATCS Course Handbook

Introduction

Aims and Objectives

The main aim of the course is to get you to think about the nature of the mind and to learn how cognitive scientists have attempted to explain cognition. There are a number of different perspectives for studying the mind and providing accounts of how it works. These include: Philosophy, Psychology, Artificial Intelligence and Linguistics. Furthermore, there are competing theories of how the mind works both across and within each of these disciplines. A main objective of the course is for you to understand how these various approaches complement and differ from each other.

The course is also designed to help you to learn skills that you can apply throughout your university life and beyond. These skills include note taking, critical reading, debating, project work, giving presentations, essay and report writing.

The course uses a variety of teaching methods and materials. Teaching will be through a combination of integrated lectures and seminars. The lectures will introduce and illustrate concepts, theories and techniques that are central to cognitive science. The seminars will provide you with the opportunity to discuss issues like whether the mind can be likened to a computer and whether computers can think. In the lectures you should note points to raise in the forthcoming seminars: these should include anything you would like to discuss further.

Lecture times and topics

• ATCS lecture, Tuesday 12.30-13.20 in C133

• Professor Margaret Boden's Open Lecture, Tuesday (but missing week 6 and 10) 11.30-12.20 in C133.

Attendance at the ATCS lecture is compulsory. Attendance at Prof. Boden's open lecture is highly recommended but may not be possible in all cases.

Lecture topics

The topics for the ATCS lectures are below.

• Week 1 (lec01) Introduction to Cognitive Science

• Week 2 (lec02) Mental Representation

• Week 3 (lec03) Approaches to Vision

• Week 4 (lec04) Seeing

• Week 5 (lec05) Learning and Memory

• Week 6 (lec06) Language

• Week 7 (lec07) Conversation

• Week 8 (lec08) Imagery

• Week 9 (lec09) Consciousness

• Week 2, Man as Machine: Origins of the Idea

• Week 3, Are Minds Machines Too?

• Week 4, The Rise of Computational Psychology

• Week 5, The Birth of Connectionism

• Week 7, When GOFAI was NEWFAI

• Week 8, A-Life in Embryo

• Week 9, Philosophies of Mind and Machine

Week 1 - Introduction

Week 2 - Representation

Week 3 - Vision

Week 4 - Seeing

Week 5 - Learning

Week 6 - Language

Week 7 - Conversation

Week 8 - Imagery

Week 9 - Consciousness

The ATCS `book' - the whole website in one, printable document

Seminars

The first half of each seminar is primarily activity-based, e.g. carrying out various exercises related to the lectures and planning your assignments. The second half provides you with an opportunity to discuss topics and issues that have arisen in the lectures and your readings for each week. (See next section for debating topics.)

Where the scheduled seminar activity does not fill the entire period, the rest of the time may be spent on the relevant debate exercise. The debate topics may be accessed via the `week' sections of the handbook.

Tutors should run these debates as follows. First, divide the seminar group into two halves, delegating one half to argue in favour of the motion and the other against it. Allow approximately half an hour for the debate to be carried out, making sure that individuals from both sides get a reasonable chance to make their points. At the end, write down the names of those aguing `in favour' on one sheet of paper and the names of those arging `against' on another.

Circulate the `in favour' list around the `against' group and vice versa, asking each person to (secretly) record their votes for the best performers from the other group. A `p' should be placed next to the name of the most persuasive debater; an `a' should be placed next to the name of the most authoritative (i.e., best read) debater and a `c' should be placed next to the name of the most cooperative (i.e., best team-working) debater.

Finally, collect in the two pieces of paper and announce the results of the voting, adding in your own feedback as appropriate.

Make sure to remind everyone that this assessment is informal and will not affect degree results.

Assessment

There are 3 assignments:

1. a project report (counts for 40% of final mark)
2. a presentation (counts for 10% of final mark)
3. an essay (counts for 50% of final mark)

Please apply to the COGS school office for their latest instructions regarding the submission of assessed coursework. In the year 2001-2002 the system involved submitting work in the foyer of the COGS building between 2 and 4pm on Thursday of the relevant week. Please make sure that you put your name and major on your work and place it in the appropriate submission box. The written assignments should be typed or word processed (unless special permission has been given).

Assignment 1: Mental Models Project

(i) how a piece of technology works (e.g. mobile phone, computer, internet, telephone, library system);

(ii) a physical location (e.g. university campus, COGS, the library, halls of residence, Brighton).

Stage 1: Finding out about mental models

To study mental models you will need to `interview' 3 people (who are not on the ATCS course), using a techique or techniques you consider most appropriate. Your goal is to try to uncover the following:

• the subjects' mental model of how the technology works in terms of functional aspects
• the subjects' mental model of the spatial layout of the place in terms of structural aspects

Stage 2: Representing mental models as a scientific explanation of how the mind works

You will then need to consider the `raw' data you have collected in terms of how each of the above is represented in the mind (e.g. rules, mental images, propositions, mental animations).

You will need to analyse the data collected and discuss it with respect to a theoretical perspective. In particular, you will need to discuss the difference between different kinds of models and to what extent these can be considered as scientific explanations of what your subjects have told you.

Resources

You should read up about mental models, mental representations and everyday vs scientific explanations.

Particularly useful is Don Norman's paper on mental models (extracts available online). Also of relevance is the chapter from Preece et al.'s book on Human Computer Interaction, extracts also available online.

See also the following.

• Preece, J., Rogers, Y. et al (1994) Human Computer Interaction. (Ch. 5: Knowledge and Mental Models.)
• Johnson-Laird, P. (1983) Mental Models. (Ch. 7, 15)
• Gentner, D. and Stevens, A. (1983) Mental Models. (Ch. 1)
• Aitkenhead, A.M. and Slack, J.M. (1987) (Eds) Issues in Cognitive Modelling. (Ch. 4)
• Rogers, Y., Rutherford, A. and Bibby, P. (1992) Models in the Mind. (Part 5)
• Semin, G. and Gergen, K. (1990) Everyday Understanding (Ch. 1)

A 1500 word write-up of the project is required. It should include:

• Title.This should be a specific title describing your project. It should NOT simply be entitled `mental models project'.
• Introduction. Explain what mental models and mental representations are (showing evidence of reading the relevant literature).
• Aims and objectives. Describe the purpose and goals of your project.
• Method. Describe the methods you have used to elicit the mental models.
• Results. In this section you should present your findings in terms of the kinds of mental models elicited (e.g. structural, functional). In reporting your findings you should describe any commonalities and individual differences across and within subjects. To illustrate your findings you should also include examples of actual data.
• Discussion and Conclusions. You need to discuss your findings at a theoretical level. Do your findings support a particular theoretical claim about the nature of mental models? Do they suggest how knowledge is represented in the mind? If so, how? Other issues to discuss:
• how representative are your findings in relation to how the mind works?
• how much knowledge remains uncovered (i.e. unavailable to conscious inspection).

  Finally, you should discuss how effective your methodology was in eliciting mental models and what further research you could do to find out more about the nature of mental models.

This assignment counts for 40% of the marks for the course.

Assignment 2: Presentation

This assignment counts for 10% of the marks for the course.

Assignment 3: Essay

Cognitive science treats the mind as a machine and thus seeks to express theoretical ideas as computational models that generate the behavior. ... The process of model building encourages a deep understanding of the theory." (D.W. Green & others, Cognitive Science: An Introduction, Blackwell, 1996, p. 19.) Discuss this view in the context of the general methodology of Cognitive Science, and comment on whether the paradigm it describes is valuable in the study of ONE of the following: Psychology OR Artificial Intelligence OR Linguistics OR Philosophy.

This assignment counts for 50% of the marks for the course.

Reading List

• Green, D. W. and others (1996) An introduction to Cognitive Science. Blackwell Publishers.

A more recent textbook which may also be of help is

• Ernest Lepore and Zenon Pylyshyn (1999) What is Cognitive Science?

• Robert M. Harnish (2002) Minds, Brains, Computers.

Another useful resource http://www-psych.stanford.edu/cogsci.html

Other useful books include:

• Boden, M. A. (1990) The Creative Mind. Abacus paperback.
• Boden, M.A. (1986) Artificial Intelligence and Natural Man. Harvester Press and Basic Books, 1977. Second edition 1986.
• Bolter, D. (1984) Turing's Man: Western Culture in the Computer Age. Duckworth.
• Dennett D.C. (1978). Brainstorms:Philosophical Essays on Mind and Psychology. Cambridge, MA: MIT Press.
• Dreyfus, H.L. (1979) What Computers Can't Do (Revised edition). Harper and Row.
• Gardner, H. (1985) The Mind's New Science: A History of the Cognitive Revolution. Basic Books (reprinted 1987). [He has written several books on Cognitive Science. Look for others by him.]
• Garnham, A. (1991) The Mind in Action: A Personal View of Cognitive Science. Routledge.
• Haugeland, J. (1981) Mind Design: Philosophy, Psychology and AI. MIT Press.
• Hofstadter D.W. (1979) Godel, Escher, Bach: An Eternal Golden Braid. Harvester Press and Penguin books.
• Hofstadter D.W. and Dennett D.C. (Eds.) (1981) The Mind's I: Fantasies and Reflections on Self and Soul. Harvester Press.
• Johnson-Laird, P.N. (1993) The Computer and the Mind 2nd Edition. Fontana.
• Norman, D. (1980) Twelve Issues for Cognitive Science. Cognitive Science 4, 1-33.
• Osherson, D.N. and E. E. Smith (Eds.) (1990) An invitation to Cognitive Science. MIT Press.
• Posner, M.I. (Ed.) (1989) Foundations of Cognitive Science. MIT Press.
• Pylyshyn, Z.W. (1984) Computation and Cognition: Towards a Foundation for Cognitive Science. MIT Press.
• Searle J.R. (1980) Minds, Brains and Programs [with peer commentaries]. Behavioral and Brain Sciences 3, 417-457.
• Searle J.R. (1984) Minds, Brains and Science: The 1984 Reith Lectures. BBC Publications.
• Stillings, N.A., Feinstein, M.H., Garfiled, J.L., Rissland, E. L. Rosenbaum, D.A., Weisler, S.E. and Baker-Ward, L. (1987) Cognitive Science: An Introduction. MIT Press.
• Wilson, R. and Keil, F. (Eds.) (1999) The MIT Encyclopedia of the Cognitive Sciences. MIT Press.

Additional reading for consciousness

Additional reading for mental models

See also

• Anderson, J. R. (1978) Arguments concerning representations for mental imagery. Psychological Review 85, 249-277.
• Borgman, C. L. (1986) The user's mental model of an information retrieval system; an experiment on a prototype on-line catalogue. International Journal of Man-Machine studies 24, 47-64.
• Briggs P. (1988) What we know and what we need to know: the user model versus the user's model in Human Computer interaction. Behaviour and Information Technology 7, 431-442.
• Garnham, A. (1987) Mental models as representations of discourse and text. Ellis Horwood.
• Gilhooly, K. J., Keane, M. T. G., Logie, R. H. & Erdos, G. (Eds.) (1990) Lines of thinking. Volume 1. Wiley. (Chapter by Byrne & Johnson-Laird.)
• Kieras, D. E. & Bovair, S. (1984) The role of mental models in learning to operate a device. Cognitive Science 8, 255-274.
• Morris, P. (Ed.) (1987) Modelling Cognition. Wiley.
• Paivio, A. (1986) Mental representations: A dual coding approach. Oxford University Press.
• Payne, S. J. (1991) A descriptive study of mental models. Behaviour and Information Technology 10, 3-21.
• Posner, M. (1989) The foundations of cognitive science. MIT Press. (Chapter by Johnson-Laird.)
• Rouse, W. B. & Morris, N. M. (1986) On looking into the black box; prospects and limits in the search for mental models. Psychological Bulletin 100, 349-363.
• Sharkey, N. E. (Ed.) (1989) Models of cognition: A review of cognitive science. Volume 1. Ablex.
• Vosniadou, S. & Brewer, W. F. (1992) Mental models of the earth: a study of conceptual change in childhood. Cognitive Psychology 24, 535-585.
• Wilson, J. R. & Rutherford, A. (1989) Mental models: Theory and application in human factors. Human Factors 31, 617-634.

ATCS (lec01) Introduction to Cognitive Science

The study of the mind

• One of the greatest intellectual challenges attacked from a vast array of different directions.

• Cognitive Science seeks a scientific understanding of the mind.

• We cannot see our minds working, so we must make inferences from indirect evidence.

What does the mind do?

• Perceiving - learning - remembering
• Controlling actions
• Planning - imagining - creating
• Understanding others - communicating with others
• Making decisions - solving problems
• Daydreaming

Everyday explanations

• We try to explain the actions of others and ourselves all the time by referring to their intentions, beliefs, emotions etc.
• Thus social cognition requires us to try to understand the mental states of others.
• But pop psychology has limits

Limitations of pop psychology

• Pop psychology accounts can be inconsistent, unreliable, specific and not predictive;
• they do not deal with many essential cognitive processes - such as how we see, how we speak, how we read.
• In short, it is easy to mislead ourselves about how our minds work.

Towards a science of mind

• A search for principles which can provide general, reliable accounts with predictive power.
• A basis for practical applications:

  - better understanding of learning may give better teaching methods; - intelligent computers may provide valuable services.

Tools for the study of mind

• need a language to give a common way of conceptualising phenomena;
• a framework in which to develop theories (we cannot hope to explain everything at once);

Methods for studying mental phenoman: 1. Language

• Metaphors may help:

  - the mind as machine - the mind as symbol system

• Mental representations and their nature are central to much theorising - yet even their existence is hotly debated!
• Mental processes are taken to act on representations.

2. Framework

 Biological level   neurons and the like
       ||
       \/
 Cognitive level    mental representations and processes
       ||
       \/
Behavioural level   performance of task

3. Methods

• introspection

  - think-aloud studies

• questionnaires and interviews

  - experiments - computer simulations

What Characterises Cognitive Science?

The analysis of particular cognitive abilities, e.g. Problem solving, language.

An interdisciplinary approach.

Seminar materials

ATCS: Seminar for `Introduction to Cognitive Science' lecture

Objectives

Note that general instructions for seminar and debate exercises are available under the heading `Seminar sessions' on the website or in the ATCS `book'.

Instructions to tutors

The objective is to get them thinking about everyday explanations of the mind, cognition etc. and how some things we take for granted (e.g. the fact someone reads - we don't ask how they achieve this) whilst other thing we question (e.g. why the daughter gave Dad his favourite jam for breakfast, and her motives).

Ask them what are the limitations of everyday explanations of the mind and then what might be the benefits of a scientific explanation. Ask them to explain the taken for granted aspects of cognition in everyday explanations (e.g. how we see, how we read, how we speak) and how sci- entific explanations have tried to explain these largely unconscious processes. Maybe discuss the difference between conscious and unconscious cognitive processes; maybe also discuss lev- els of explanation (see first chapter of Green and others).

If there is time, repeat the role-playing exercise with different students. Get them to point out any new observations about the scenario.

Follow-up work

Scenario from Green et al.

This is a fictional conversation over breakfast between two adults, Mom and dad, their teenage daughter, Lucy, and, their two and a half year daughter, Zara. It is set in a kitchen/living room. The phone rings.

Mom picks up the phone, listens, and says: I'll get her. How's the coffee?

Lucy: Started. Lucy goes to phone.

Zara in high chair at the table, reaches forward toward milk carton, saying: more milk mommy!

Dad: Here's some milk.

Zara turns towards the cupboard, points and says, Mommy, - ops!

Mom: Hm! What darling?

Zara: Pops! Pops!

Mom: Oh, can you get her some - I'm seeing to the toast.

Lucy: I'm eating at Jane's tonight, Mom - OK?

Dad: I thought it was your homework evening. Door bell rings.

Lucy: I'll go.

Mom frowning; She's worked every night, John.

Dad: She didn't work last night. She sat in front of the TV.

Lucy: For you, from Gran I think.

Mom struggling to open the parcel.

Zara: Gran send me bear.

Mom That's right! Granny sent you your bear for birdthday, last month, didn't she?.

Mom goes out to the living room.

Dad: Toast! Lucy! It's Edna in here.

Lucy: I'll make some more. Lucy discovers the toaster is broken. She makes toast heating the bread in a frying pan..

Zara: bangs the table Allgone! Gone!

Dad: Yes, you've eaten it up ... Cleared the bowl. Lucy are you pouring the coffee? Where's the butter? Can you bring it over? And the jam.

Lucy: Anyone seen the tray?

Dad: It's not over here.

Mom comes back with scissors and opens the parcel.

Mom: It's a jumper for Zara. Look, won't she look great! Mom puts down the jumper and starts to read the letter enclosed. She says she has fixed up something ... What's this? `Nolinay?' Ah! `Holiday'! Granny's writing doesn't improve. `I've fixed up a holiday with friends who have a yacht. We're sailing to the Polish port of (she spells out the letters) SZCZECIN. I've never been sailing before.' Where did she go last year? I've completely forgotten.

Lucy: Mom, have you seen the tray?

Dad: The Grand Canyon. I remember the postcard she sent us. The cat comes in, stretches, and lies down.

Mom: I just saw it in the living room.

Lucy comes across with the toast, butter, jam, coffee pot, and mugs on the bread board.

Dad; Ah you litttle monster! My favourite jam. You still have to work tonight!

ATCS: Debate for `Introduction to Cognitive Science'

Instructions to tutors

Motion

• Everybody thinks they can explain other people's behaviour. But these explanations are unscientific and therefore wrong.

ATCS (lec02) Mental Representation

What is a representation?

• map of sussex campus?
• sussex crest?
• photo of sussex library?

Representations

Criteria

• A representation is something that stands for something else.
• It must convey some aspects of the represented world.

Palmer's features

1. what the represented world is
2. What the representing world is
3. What aspects of the represented world are being modeled
4. What aspects of the representing world are doing the modeling
5. What are the correspondences between the two worlds

Representational system

• For example, we might represent a big lion as a vertical thing of a certain length (and a small tortoise as a much smaller vertical thing).
• But what about abstract things such as relationships (e.g., taller-than)?

Mental Representations

• What features do they have?

• represented world is brain states
• representing world is theoretical structures
• examples of theoretical structures include mental images, mental models, rules, analogies, concepts, schemas

Theories of mental representation - (i) mental images

• Not pictures in the mind that are read by a little person inside the head (the homunculus fallacy).
• Not an infinite number of photos stored away in a library (the slide projector model).

So what are they?

• Images are generated, dynamic and transient.
• Images are used in a whole range of mental activities (solving problems, memorising information, daydreaming).

Theory of mental images

• Kosslyn (1983) has developed a theory of mental imagery which uses the ``brain is like a computer'' metaphor.
• Mental events (images, thoughts) are conceived as corresponding to the functional operations of a computer.
• Images can be manipulated in a computer as symbols are, with operations such as rotation and deformation, and then displayed.
• But who watches the display?
• No-one, since mental images are not pictures but symbols in a matrix.

Theories of mental representations - (ii) Propositions

• Most cognitive scientists have focused on theorising about propositional representational systems.

• Knowledge about the world is assumed to be represented as a set of discrete symbols. - These include relations (e.g. on, near, besides)

• Knowledge is stored in terms of concepts, categories and properties of these. - e.g. cat: animal, pet, furry, 4 legs, tail, purrs.

• Concepts are represented by formal statements. - e.g. is-a(cat, animal), under(cat, table), has(cat, 4 legs) - On(c1, t1) [cat c1 is on table t1]

Other kinds of knowledge representation

• e.g. schemas, rules, semantic nets

• Declarative is assumed to be represented as a formal proposition, e.g. rule.
• Procedural is assumed to entail an active process or procedure.

Example: semantic nets

References

• Aitkenhead, A.M. and Slack, J.M. (1985) Issues in Cognitive Modelling. Chapters 2, 3, 4.

• Kosslyn, S. (1983) Ghosts in the Mind's Machine.

Seminar materials

ATCS: Seminar for `Mental Models and Representation' lecture

Introduction

Warm-up exercise: Students should look around the room for one minute and then, keeping their eyes closed, do the following.

• to count the number of windows in the room;
• to count the number of people in the room;
• to remember who the people are.

Following this, students should examine (and discuss) the `raw data' (diagram/sketch/prose) and the mental representations they used (mental images/mental models etc.) to come up with their answers. E.g. Did they `run' a mental model, going round the room and mentally counting the number of windows? Students should try to identify the difference between a mental model and a mental image.

Project

Introduce the students to the project (see course handbook). Ask them to think about the difference between structural and functional models. Get them to discuss the differences between different methods for eliciting mental models and the pros and cons of the following techniques:

• think aloud;
• explaining whilst doing (e.g. using a diagram or piece of equipment);
• writing and drawing;
• questionnaire;
• probing interviews.

• psychology of language approach (e.g. Johnson-Laird);
• HCI/cognitive science approach (e.g. Norman, Rogers).

Work for the following week

ATCS: Debate for `Mental Representation' lecture

Motion

• Cognitive science is important and valuable because it paves the way for intelligent computers which will fulfil the roles so poorly played by doctors, bank managers, teachers etc.

ATCS (lec03) Approaches to Vision

Vision and the study of mind

• our physical environment - think of walking along a clifftop, or playing a ball game
• our social environment - think of going into a club or café
• our `information environment' - think of looking at a diagram in a book, or on a web page

Visual representations

• beneficial, even essential, for some kinds of communication - why do knitting patterns and recipes have pictures? - why do textbooks have diagrams when they could use words? - why do writers use visual metaphors?
• central to reasoning processes - can you work out how many different combinations of coins make 7p without using a mental picture of some kind?
• central to memory - what was your first day at school like?

Visual processing

• uses a large part of the brain
• is extraordinarily effective
• is cognitively impenetrable
• is extremely hard to analyse, understand or imitate

Questions about vision

• images on our retinas?
• light?
• what we expect?
• our mental models?
• a distorted version of reality?
• 3-D surfaces and objects?
• possibilities for action?

How is visual processing organised?

• `bottom-up'? - images are analysed in a pipeline of processes to identify 3-D objects, their shape and position, with the least possible commitment to what they might represent
• `top-down'? - we start from hypotheses about what we are looking at; these are checked against the data coming from our visual organs in order to verify, discard or modify them

Are our visual processes modular?

• separate modules for, e.g. stereoscopic vision, colour, motion, with well-defined connections passing information between them?
• a complex network, not readily separable into modules, with multiple connections between and multiple functions for elements?

Are our visual processes specialised or general?

• can we see anything, within the physical limits of our visual systems? - we adapt to strange environments - underwater, space - and can see the structure of novel objects e.g. sculptures
• are there processes tuned to special things in our environment? - faces are special - looming is special

How can we investigate vision?

• experiments in the laboratory - probing the black box with carefully constructed stimuli - visual illusions and the limits of the system
• investigations on animals - anatomy, physiology, behaviour
• observations of real behaviour - the system doing what it evolved to do
• computational modelling - do our theories really work?

Can we make machines that see?

• philosophical questions - could a robot `really' see?
• technical questions - what algorithms? what architecture?
• financial questions - who will pay?
• and will it tell us anything about how we see?

How do you cross the road?

Maybe some of the following ...

• Where is the kerb? Where are the parked cars, the road markings, junctions? Which direction takes me straight across the road?

Jobs for the visual system to do

• segment the image into meaningful regions, perhaps using boundaries of brightness, colour, texture
• recognise objects, perhaps using some kind of template
• find the position and orientation of surfaces relative to you - hence e.g. the direction perpendicular to the kerb line, whether the road surface is level
• scan the scene using head and eye movements, and integrate the information obtained from different fixations

The dynamic environment

Your visual system might

• estimate speeds, directions of motion and positions of moving cars

• estimate the time to collision of approaching vehicles

Optic flow

What you can do with optic flow

There is thus a computational theory for at least part of the visual control of road crossing.

There is also some experimental evidence that the visual system makes use of this kind of information:

• infants sensitive to looming
• gannets plummeting
• people punching balls

J.J. Gibson and "nouvelle AI"

• optic flow and texture gradients were developed as specific examples of powerful sources of visual information which had been largely neglected
• invariants - perceptual properties of objects which are independent of viewing position - are seen as key to reliable visual information pickup
• visual perception picks up affordances - properties of the relationship between an object and the observer, specifying potential interactions - not abstractions such as shape or identity
• his theory of direct perception rejects a `hypothesis testing' approach to vision in favour of a view in which information specified directly in the optic array is used to control action

However, practical image understanding technologies exploit many of the tools developed in computer vision over the past 4 decades.

Conclusion

Some processes are understood; but the overall architecture and functioning of the human visual system remains largely uncertain.

The challenge can be met only with a wide variety of approaches: computational, psychological and philosophical; experimental and theoretical.

References

• Green & others (course textbook), Chapter 4

• Sharples, M. et al., Computers and Thought (MIT Press, 1989), Chapter 9
• Bruce, V., Green, P. & Georgeson, M., Visual Perception: physiology, psychology and ecology (Psychology Press, 1996)

• Gibson, J.J., The Senses Considered as Perceptual Systems (Allen & Unwin, 1968; Waveland Press, 1983)
• Marr, D., Vision: A computational investigation into the human representation and processing of visual information (Freeman, 1982)

Seminar materials

ATCS: Seminar for `Approaches to Vision' lecture

Topics for discussion

How has our visual system evolved? What functionality must it have to have in common with the visual systems of e.g. insects, what must be different?

What is a visual system? Eye, retina, brain (a lot of it) in our case; camera, light-sensitive array, computer+programs in a computer's case. Maybe discuss what it would take to simulate vision using a computer, particularly if you have a significant number of AI students.

Exercise on 3-D vision

• texture gradients (things get smaller as they get further away)
• position in image (when there's a ground plane, nearer top of picture is further away)
• linear perspective (fans of lines in image imply receding parallel lines in space)
• size of known objects
• haze in the distance
• light and shade - brighter surfaces oriented towards light source

How does perception of 3-D shape from a picture differ from perception of the shape of a real object. Remember that in real life we also obtain information about shape from

• stereoscopic vision (combining the images from our two eyes)
• motion (head and body motion gives us a changing view of a scene)

Final questions

ATCS: Debate for `Approaches to Vision'

Motion

• The fact that we `know what we are looking at' means that the brain must construct an internal model of the external reality.

ACTS (lec04): Seeing (Boden)

Introduction

Slide 1

Slide 2

Slide 3

Slide 4

Slide 5

Slide 6

Slide 7

Slide 8

Slide 9

Slide 10

Slide 11

Slide 12

Slide 13

Slide 14

Slide 15

Slide 16

Seminar materials

ATCS: Seminar for `Seeing' lecture

Introduction

"Seeing" activity (MB)

Seeing is believing

Does this mean:

(i) Use your eyes! If you see it, you're justified in believing it.

or:

(ii) Seeing is a cognitive activity. It's not a passive registration of data.

Mental models project part 2

Go round the class and ask students to talk through their experience of collecting data: what they found (especially information they did not expect), what problems they experienced etc.

Ask them what the limitations of using verbal protocols are, e.g.:

(i) incomplete/partial data

(ii) incorrect data - people may say things (and believe them) but act in a different way

(iii) people may say things to please you even though they may never have thought about them before.

Then get them to talk through how they would explain their findings at a theoretical level. Do their data suggest how this kind of knowledge is represented in the mind? As propositions or images or both? Are other representations involved (e.g. semantic networks)?

Finally, discuss how to write up report and remind them of the deadline (see the course handbook).

ATCS: Debate for `Seeing'

Motion

• Seeing isn't done by the eyes, but by the brain.

ATCS (lec05) Learning and Memory

Exam

Slide 1

Slide 2

Slide 3

Slide 4

Slide 5

Slide 6

Slide 7

Slide 8

Slide 9

Slide 10

Slide 11

Slide 12

Slide 13

Slide 14

Slide 15

Slide 16

Slide 17

Slide 18

Slide 19

Slide 20

Slide 21

Slide 22

Slide 23

Slide 24

Seminar materials

ATCS: Seminar for `Learning and Memory' lecture

Topics for discussion

Imagery is just a personal experience. Discuss in the light of current research

Is imagery best conceptualised as image (e.g., .gif) files?

What does Imagery tell us about models of memory?

Exercise

• What were you doing when you heard about the trade centre bombing?

ATCS: Debate for `Learning and Memory'

Motion

• Cognitive science is an applied, as well as an explanatory science. It can be used to solve hard problems in the real world.

ATCS (lec06) The structure of language

Lecture Summary

I then present the notion of the lexicon and lexical items. Lexical items are principally words; I introduce the notion of word-classes (parts of speech) and show what sorts of information needs to go into the entries of lexical items.

I then introduce the notion of linguistic rules, both in the lexicon and in grammar. These are recurrent patterns, not instructions about how to behave. I concentrate on the notion of constituency and show how you can test whether some string of words is a constituent of a higher unit.

I end (if there's time) by asking where these structures and rules are, and how they get there.

Reading

(** are the easiest STARTING points. Don't also FINISH with one of these!)

• ** Aitchison, Jean (1987) Teach yourself linguistics. Hodder and Stoughton, third edition. [Parts I and II and chapter 11. Note that library catalogue records title as simply Linguistics.]
• Aitchison, Jean (1989) The articulate mammal. Unwin Hyman, third edition.
• Aitchison, Jean (1994) Words in the mind. Blackwell, second edition. [Part I and chapters 9 and 11.]
• ** Fromkin, Victoria and Robert Rodman (1998) An introduction to language. Harcourt Brace, sixth edition. [Chapters 1, 3, 4 and pp. 328-45, 350-8.]
• Hudson, Grover (2000) Essential introductory linguistics. Blackwell. [Esp. pp. 57-150.]
• Lyons, John (1968) Introduction to theoretical linguistics. Cambridge University Press. [Esp. pp. 1-3 and part II.]
• Lyons, John (1981) Language and linguistics. Cambridge University Press. [Esp. chapters 1, 4 and 8.]
• Pinker, Stephen (1994) The language instinct. Penguin. [Esp. chapters 1-5, 10, 12.]
• oo Trask, R.L. (1999) Language: the basics. Routledge, second edition. [Esp. chapters 2, 3 and 8.]
• Wardhaugh, Ronald (1993) Investigating language. Blackwell. [Esp. chapters 1, 3, 4, 7 and 8.]
• ** Yule, George (1996) The study of language. Cambridge University Press, second edition.

• Trask, R.L. (1999) Key concepts in language and linguistics. Routledge.

Seminar materials

ATCS: Seminar for `Language' lecture

Objectives

Discussion

Some key ideas are:

• some linguistic elements are universal (i.e. occur in all languages)
• some linguistic elements are implied by others (if a language has one, it'll have some other one too)
• some linguistic elements are not universal, and must inferred by learners from data they hear around them
• some linguistic elements have structure (can be broken down, analysed)
• some linguistic elements seem to be processed as if they didn't have structure even if they appear to be analysable (stock phrases like _don't know_, _see you later_, _good on you, mate_); they behave as lexical items
• rules of language can be viewed as processes or procedures, and lexical entries as places where (unpredictable) information is stored
• rules of grammar allow you to construct and understand sentences which you haven't come across before, so languages are larger than the sum of their speakers' experiences
• languages are open-ended and need special kinds of rules to describe them

(1) What is a linguistic rule? How do prescriptive and descriptive rules differ?

(2) How is the lexical entry for certain words structured; e.g. SIGH, THINK, COW, SHEEP?

(3) What is the evidence that human beings operate with linguistic rules in acquiring and processing language?

(4) How can you show that human beings, when speaking, sometimes pull stored items out of their lexicon directly, and sometimes construct what they say in systematic ways, using these stored units to build bigger units?

ATCS: Debate for `Meaning and Conversation'

Motion

• Speakers regularly and intentionally refrain from saying what they really mean in service of the higher goal of politeness in its broadest sense, that is, to fulfill the social function of language.

ATCS (lec07) Conversation

Overview

1. Introduction
2. The organisation of conversation: turn-taking
3. Meaning in conversation and the construction of topic
4. Variation in conversational behaviour

Introduction

• How easy is it to have a conversation?
• Factors which influence conversational interaction: e.g. context, participants, topic of discussion.
• The rules which govern conversational behaviour: characteristics of the art of conversation: e.g. who may speak, when, and on what topic.

The organisation of conversation: turn-taking.

• No gaps, no overlaps.
• How do we know when we can take a speaking turn? What clues do we use to identify when the current speaker will finish?
• Transition Relevance Places and their identification in the flow of conversation.

Sacks et al. rules

• Rule 1. Current speaker selects next
• Rule 2. Self-selection
• Rule 3. Current speaker continues

Speaking 'out of turn'.

• Some such speech is crucial to conversation. Schegloff (1980) argues that assent terms (otherwise known as minimal responses or back channel speech) are a necessary prerequisite to successful conversation.
• Other 'out of turn speech' may be perceived as signalling a breakdown in conversational flow: e.g. interruptions. When do we feel that we have been interrupted? The relationship between interruption and topic of conversation

Meaning in conversation and the management of topic

• How do we construct meaning in conversation and how do we know that we agree on the topics and themes of interaction?

• The Co-operative Principle and Grice's Maxims of Conversation

• The Co-operative Principle:

How this principle relates to coherence in social discourse and to the construction and organisation of topic: e.g. topic coherence, topic drift, topic conflict.

Grice's Maxims

• Maxim of Quantity: make your contribution as informative as required.
• Maxim of Quality: do not say what you believe to be false or that for which you lack adequate evidence.
• Maxim of Manner: avoid obscurity of expression and ambiguity; be brief and orderly.
• Maxim of Relation: be relevant (see Sperber and Wilson's theory of Relevance)

Variation in conversational behaviour

• No Gaps, No Overlaps: cultural and social differences

Deborah Tannen's (1989) analysis of the 'high intensity style' of the New York Jewish community.

Jennifer Coates' (1996) research on male-female differences in overlapping speech : the competitive style of men, and the co-operative style of women.

Reading:

• Green, D. et al Cognitive Science. Chapters 8 and 9.

• Downes, W. (1984) Language and Society. London: Fontana. Ch.8.
• Wardaugh, R.(1985) How Conversation Works. Oxford: Blackwell.

• Coates, J. (1996) Women Talk. Oxford: Blackwell
• Grice, P. (1975) 'Logic and Conversation'. In P. Cole and J. Morgan (eds.), Syntax and Semantics, Vol 3: Speech Acts (pp. 225-42). New York: Academic Press.
• Sacks, H. Schegloff, E. and Jefferson, G. (1974) 'A simplest systematics for the organisation of turn-taking in conversation'. Language 50, 1974, 696-735.
• Sperber and Wilson (1986) Relevance: Communication and Cognition. Cambridge, MA: Harvard University
• Tannen, D. (1989) Talking Voices: Repetition, Dialogue and Imagery in Conversational Discourse. Cambridge: Cambridge University Press.

Seminar materials

ATCS: Seminar for `Meaning and Conversation' lecture (Woods)

Suggested themes for discussion:

Which contextual features influence conversational discourse?

How useful is the concept of Adjacency Pair in the analysis of conversation?

How are turns allocated in conversational interactions?

What happens if turn-taking rules break down?

How important are assent terms in conversational interaction?

Do all conversations follow the Principle of Co-operation?

How are themes and topics introduced into conversation?

When is making conversation difficult? What makes it so?

How does culture influence conversational behaviour?

Do all social groups converse in the same way?

Analysis.

A: I mean, do you absolutely have to use this version of the report on Monday?

B: I would like not to have to hand out that version because you can see what it looks like.

A: You've stopped working on it now because you have other things to do, haven't you? Would you be able to carry on with it on Monday morning?

C: Where are you taking it?

(pause)

C: Are you taking it anywhere interesting?

B: London.

C: Oh right.

How is topic managed in the following conversation?

How useful are Grice's Maxims of Conversation for the explanation of the construction of meaning in conversation? e.g. The maxim of Quantity: make your contribution as informative as required.

Speaker A: Shall we go out for supper tonight?

Speaker B: Janice is arriving at Heathrow

Does speaker B follow the Maxim of Quantity? Give examples of question-answer (adjacency) pairs in which answers to questions follow and do not follow this maxim.

Maxim of Relation: be relevant

Speaker A: I like your shoes

Speaker B: I didn't steal them

Is B's answer relevant or not? What is implicated by B's answer and why?

ATCS: Debate for `Meaning and Conversation'

Motion

• Speakers regularly and intentionally refrain from saying what they really mean in service of the higher goal of politeness in its broadest sense, that is, to fulfill the social function of language.

ATCS (lec08) Imagery (Langham)

Note

Slide 1

Slide 2

Slide 3

Slide 4

Slide 5

Slide 6

Slide 7

Slide 8

Slide 9

Slide 10

Slide 11

Slide 12

Slide 13

Slide 14

Slide 15

Slide 16

Slide 17

Slide 18

Slide 19

Slide 20

Slide 21

Slide 22

Seminar materials

ATCS: Seminar for `Imagery' lecture (Langham)

Topic for discussion

• How is all we have learnt this term stored? As images (cf. the Vision lecture) or as words (cf. Language lectures).

Reading

You may also wish to consider Matlin, M. 1989 Cognition 2nd edition Holt, Rinehart & Winston Ch. 5 has a good overview of the issues

Kosslyn. S,M,. 1975 Information representations in visual images Cognitive Psychology 8 341-370

Kosslyn SM 1981 The medium and the message in mental imagery : A theory Psychological review 88 46-66

ATCS: Debate for `Imagery'

Motion

• Imagery is of no use in rational thought.

ATCS (lec09) Consciousness and Cognitive Science

Aims of today's lecture

(b) to elucidate the supposed first-person nature of conscious experiential states (qualia);

(c) to consider if consciousness can be explained in scientific terms;

(d) to examine some of the major current theories (empirical and philosophical) attempting to account for consciousness;

(e) to look at some puzzles that have been raised in relation to materialist theories

CONSCIOUSNESS - INTIMATE YET MYSTERIOUS

• Mysteries of mind in general:
• explaining intentional content
• rationality, intelligence
• perception, sensorimotor coordination
• mind-body causation
• is mind physicalo mind = braino mind = software?
• Mysteries of consciousness:
• subjective feel - how explained?
• privacy ( first-person nature - other minds problem) - how can I know your consciousness?

CONSCIOUSNESS and MIND

• Relation between consciousness and mentality?
• Cognitive scientists and others assume that many mental processes are unconscious

The friendly black cat that we ve been looking after while our neighbours are on holiday sat peacefully on the purple mat for two hours (example from Margaret Boden)

Perceptual processing Neurotic symptoms

e.g. exaggerated fear of germs

DOES CONSCIOUSNES = MIND ?

(a) Standard cognitive science view: many mental processes are in principle unconscious (but others are actually or potentially conscious)

(b) Everything that is mental is at least potentially open to conscious awareness (John Searle, The Rediscovery of the Mind, 1992, ch 7)

(c) A more extreme view: nothing is in the mind unless it occurs directly in consciousness (Galen Strawson, Mental Reality, 1994)

(d) Another extreme: It is the result of thinking, not the process of thinking that appears spontaneously in consciousness No activity of mind is ever conscious. (George Miller, 1962)

CONSCIOUSNESS and COGNITION

(The idea that consciousness may emerge from sufficiently complex systems [ the Internet wakes up ] )

(B) Consciousness as fundamentally distinct from cognition.

Private (1st-person, not 3rd-person)

Subjective point of view

ineffable
Privileged-reporting

(C) Access - consciousness versus phenomenal consciousness

ACCESS-CONSCIOUSNESS versus PHENOMENAL CONSCIOUSNESS

• See Ned Block, On a confusion about the function of consciousness (BBS, 1995)
• A-consciousness
• ability of subject to report on states
• availability of states for global control
• a cognitive kind of consciousness - explainable in familiar cognitive science terms
• P-consciousness
• the felt aspect of c. states (qualia)
• non-cognitiveo maybe non-physical??
• Some challenge the distinction (e.g. Dennett)

CONSCIOUSNESS and SCIENCE (1)

• Can there be a science of consciousness?
• The reductive cascade in science:
• psychology -> neuroscience -> biology -> chemistry -> physics
• physicalism/ reductionism
• do experiential phenomena fit in here?
• Dissent: Thomas Nagel: What is it like to be a Bato (1974)
• Science gives objective accounts of nature - the processes in themselves, and not as viewed (e.g. lightning)
• So science can t explain subjectivity - what it is like to have a particular kind of experience

CONSCIOUSNESS and SCIENCE (2)

• Possible views on the relation:

(b) consciousness is explained in functional or computational terms

(c) consciousness is a type of non-physical process (not reducible to either neural nor functional processes) -and therefore cannot be proper subject of scientific inquiry;

(d) consciousness is non-physical, but is nevertheless still a fit subject of scientific inquiry (because the scope of science may include non-physical processes);

(e) consciousness is inherently mysterious;

THEORIES OF CONSCIOUSNESS IN SCIENCE

• Physicalist theories
• Neural correlates of Consciousness (NCC) (eg Crick and Koch, Edelman)
• Quantum theories (Penrose/Hameroff)
• Higher-order cognition theories (eg Baars: Global Workspace, Rosenthal: Higher-Order Thought)
• Connectionist theories (eg Aleksander)
• Non-physicalist theories
• Naturalistic dualism (eg Chalmers)
• Panpsychism (Chalmers, Velmans, some quantum theorists)
• Mysterian theories
• (Nagel, McGinn)

THE HARD PROBLEM OF CONSCIOUSNESS (D Chalmers)

• D Chalmers, Facing up to the Problems of Consciousness , 1995
• Also known as the explanatory gap (J Levine, 1983)
• Explaining phenomenal C isn t like explaining

  liquidity (intermolecular forces) genetic inheritance (DNA) learning (computational theories)

• Any PURELY PHYSICAL story must leave out the SUBJECTIVE FEEL

CONSCIOUSNESS AND COMPUTING

• Why an AI theory of consciousness doesn't seem to work, even if an AI theory of cognition does:
• Cognitive mental properties (e.g. winning at chess, understanding language) seem to be often about performance, not feeling;
• A computational simulation of cognitive intelligence is as good as the real thing (Calculator)
• This won t work for qualia (first-person experience)
• AI-simulated qualia aren t just as good as real qualia - it makes a difference (Earthquake scenario);
• But: Dennett: this supposed division (between cognitive and phenomenal) is bogus

PUZZLES (a)

• Are conscious events IDENTICAL WITH neural events?
• Leibniz s law: if a = b then they share all their properties in common. Yet conscious events seem to be quite different from brain events!
• Are conscious events CAUSED BY neural events?
• McGinn: How can technicolour phenomenology arise from soggy grey mattero How could the aggregation of millions of individually insentient neurons generate subjective awarenesso We know that brains the de facto causal basis of consciousness, but we have, it seems, no understanding whatever of how this can be. It strikes us as miraculous, eerie, even faintly comic.

PUZZLES (b)

• Explanatory gap/Hard Problem
• how can any theory of brain structure/function account for phenomenal feel?
• Why/how did C. evolve?
• Distribution of C. in natural world -
• are gorillas, fish, bacteria, etc. consciouso Non-terrestrial life-formso Artificial mechanisms (current/future)?
• Epiphenomenalism -
• does the brain cause conscious experienceso is there causation the other way round?
• Are we cognitively/conceptually equipped to understand C?
• The Binding problem
• why/how do our experiences cohere/
• Dreams, hypnosis, meditation, drug-based states, etc.
• Multiple personality - alternating streams of consc.

PUZZLES (c) Intuition pumps

• Inverted qualia red-green swapping
• Knowledge argument (F Jackson) Mary, the colourblind neuroscientist
• Possibility (Modal) argument (Chalmers) Zombie universe

• Rejection of qualia (Dennett, Consciousness Explained, 1991) The coffee tasters

ZOMBIES

• Discussed by David Chalmers, The Conscious Mind, 1996 (but it s an old argument):
• Zombie earth:
• Physically identical to our earth (molecule-for-molecule);
• Inhabitants are behaviourally and functionally indistinguishable from us (3rd-person duplication);
• But they have no first-person experience
• The aim is to show that no materialist theory could explain consciousness

What is the zombie argument trying to proveo

• 1. If consciousness could be explained in physical terms then the explanation must be able to show how the relevant physical processes NECESSITATE the conscious processes;
• 2. If one process P NECESSITATES another process Q, then one would not be able to coherently imagine a world containing P unless it also contains Q;
• 3. But a zombie universe, which is physically identical to ours, is coherently imaginable;
• 4. So consciousness cannot be necessitated by any physical processes;
• 5. So consciousness cannot be explained in purely physical terms.

Does the zombie argument worko

• Possible positions:
• (a) The zombie supposition is incoherent: any being that exhibited sufficiently fine-grained behaviour and functionality of a conscious being WOULD BE a conscious being (Dennett);
• (b) The Z world is logically possible because our concepts of phenomenology aren t the same as our concepts of neural functioning, even though these different concepts are tied to the same property - namely properties of the brain: so it s metaphysically impossible (Brian Loar)

• Thomas Nagel: Conceiving the impossible and the mind-body problem (1997)
• Our current concepts of experience and neural processing imply a logical separation;
• But in a future science, we may have reason to reconceptualize our notions of experience so that they are logically tied to the underlying neural processes (such conceptual revisions are common in science)
• So a zombie universe may seem coherent according to present concepts, but not necessarily according to future concepts

Consciousness - some key QUESTIONS

• How hard is the hard problem of consciousness?
• Is it soluble?
• Can a materialist theory work?
• Is Consciousness an appropriate subject for scientific investigation?
• Is consciousness cognitive or computationa?
• How widely is consciousness distributed in the world? (Primates? bats? insects? plants? rocks?)

Seminar materials

ATCS: Seminar for `Consciousness' lecture (Boden)

Debate

MOTION: Consciousness will always be beyond scientific understanding.

OPPOSITION: Science can explain consciousness---and it's already at least half-way there.

ATCS: Debate for `Consciousness' lecture

Motion

• Brain-imaging can't help us solve the problem of consciousness.

ATCS: Extracts from `Some Observations on Mental Models'

Introduction

This paper is very useful for the first assignment.

Plate 1

Plate 2

Plate 3

Plate 4

Plate 5

Plate 6

Plate 7

Plate 8

ATCS: Extracts from `Knowledge and Mental Models'

Introduction

This paper is very useful for the first assignment.

Plate 1

Plate 2

Plate 3

Plate 4

Plate 5

Plate 6

Plate 7

Plate 8

Plate 9

Plate 10

Plate 11

Plate 12

Plate 13

Plate 14

Plate 15

Plate 16

Plate 17