Procedure:
Collect a sample of naturally occurring and spontaneous spoken discourse or dialogue/conversation. "Naturally-occurring" can be broadly construed to include radio or TV talk shows, children's spontaneous play, spontaneous arguments or storytelling, talking about one's day among roommates, classroom discussions, task-oriented conversations in the lab, lunch conversations or monologues, etc. By "collect" we mean make video recordings (unless absolutely impossible to do and audio recording is your only option, in which case you must explain and motivate choosing this kind of data). By "spontaneous" we mean unscripted. You should collect a minimum of 15 minutes, and then transcribe at least 5 continuous minutes (usually the middle of the discourse is the most natural). By "transcribe" we mean you should make a record on paper of what you heard --a good enough record so that when we read the transcript, we can perform a pretty exact replication of the original discourse/dialogue. (read the Schiffrin appendices before doing this, and the Ochs if you have time). You will probably want to use one of Schiffrin's methods, or you will have to justify why you didn't). For this first assignment you do not need to transcribe the nonverbal behavior (that's for next week!).

Discussion:
The point is to push you to think about what discourse is and what little details escape your attention unless you are really paying attention. That may help you to think about what makes it easy or hard to model discourse in a computational system. You may want to have a particular kind of interactive system in mind when you choose your sample (a matchmaker, a language tutor, a museum guide).. Think about how a computer could replace a participant in the discourse. Another point is to think about what makes a sufficient record of discourse: how do you turn a speech event into an on-paper transcript? What parameters need to be transcribed (the words, the pronunciation of words, the intonation, pauses, false starts, etc.)?


What you must turn in:
We ask you to turn in to us (1) the typed transcript, (2) a ONE PAGE discussion of the points listed above. That is, minimally, discuss the issue of what makes an adequate transcription, and what challenges a computer might have in interacting in the discourse that you have collected.

Procedure:
Take the sample of naturally occuring spontaneous spoken discourse or dialogue that you collected for the last assignment, and transcribe 2 different channels of nonverbal behavior. For example, transcribe head nods, hand gestures, eye gaze direction, posture shifts, head tilts, or eyebrow raises. Any of the nonverbal behaviors that you have read about in the class readings is fair game. Or, if you find an article that links another nonverbal behavior to discourse or conversation, you can transcribe that. Note, however, that you should be transcribing a nonverbal behavior that co-occurs with language, and that probably has some function related to language. So, fidgeting would probably not be a good behavior to code! By "transcribe" we mean you should make a record on paper of what you saw and exactly where it occured with respect to the language you transcribed for the last time. When we read your transcript, we should be able to recreate the discourse or dialogue. A good way to do this is to put square brackets around the part of the text that the behavior occurs with, and then describe the behavior on the line below (we will hand out an example). If the behavior occurs during silence, then you might consider writing "pause" on the speech line so that you can bracket that.

Discussion:
The point is to push you to think about what nonverbal behavior is, where it occurs, and what little details about it escape your attention unless you are really paying attention. That may help you to think about what makes it easy or hard to model nonverbal behavior in a computational system. You may want to have a particular kind of interactive system in mind when you choose your sample (a storyteller, a liar, a museum guide describing 18th century techniques, or somebody listening to the previous people). Think about how a computer could replace a participant in the discourse. How could a computer recognize and understand nonverbal behavior? Or how could a computer agent produce nonverbal behavior? Another point is to think about what makes a sufficient record of nonverbal behavior: how do you turn a behavior in space and time into an on-paper set of words? What parameters need to be transcribed (the shape of the hands, the exact direction the eyes are looking, how much of the body is shifting posture, etc.)?


What you must turn in:
We ask you to turn in to us (1) the typed transcript from the previous assignment, with nonverbal behaviors (one kind per line) in between the lines of spoken transcript, (2) a ONE PAGE discussion of the points listed above. That is, minimally, discuss the issue of what makes an adequate transcription, and what challenges a computer might have in interacting in the discourse that you have collected.

Procedure:|
Think of a combination of discourse or conversation/dialogue phenomena that interests you. Perhaps what people do with their eyes with they are uttering a confirmation dialogue act. Or when smiles occur with respect to some discourse category during lectures, or during conversations. Or any other combination of two phenomena that you think might yield an interesting correlation (if you are having trouble thinking of one, ask Justine for suggestions). Take your verbal and nonverbal transcript from the last time, and code it for this co-occurence of phenomena. By "code" we mean take your transcript from the previous assignment and identify all of the instances of the two phenomena that you are interested in (for example, highlight all places where the listener looks at the speaker, or all places where the speaker utters a confirmation (use the DAMSL reading for this). Then make an excel chart that shows how many times each of your two behaviors co-occur, and how many times they occur without the other one. Have you found an interesting correlation?

Discussion
The point of this assignment is to push you to start to look at how language and nonverbal behavior function - what do they do in discourse and conversation. Once you have started to do this, you can begin to understand the literature on function, which begins next week. Functions include things like being polite, acting in accordance with cultural norms, establishing rapport, etc. And these functions are carried out in discourse and dialogue by combinations of phenomena.

What you must turn in:
We ask you to turn in to us (1) the typed transcript from the previous assignment, with coding (yellow line or whatever you like). Make sure to include a key, so we know what you are coding, and what you have used to indicate each phenomenon. (2) a ONE PAGE discussion of the points listed above. That is, minimally, discuss the issue of what you encountered when you did the coding, and what you found in your analysis of the co-occurence.

Assignment 5: A Revised Project Proposal

Procedure:
Now that you have formally heard back from us about what you want to build and how it is going to function, to do, this is where you decide what changes you need to make based on our feedback. The constraints of the assignment are the same in that your project has to rely on some theoretical basis in discourse and dialogue, and it has to be interactive. In your discussion of what you are building, make sure you show how it interacts through language and/or nonverbal behavior, and/or mediates interaction online between humans using language and/or nonverbal behavior. Include any changes that were suggested or any new ideas you have from your group's discussion of the feedback and comments. Remember that, as in the rest of the class, it is key to link theory and practice.

Your proposal should contain the following

Describe Domain and Application. Choosing a good domain is crucial for dialogue systems research. The domain should serve both to provide an interesting foundation for developing a research application and to constrain the task for which your application is designed.

Motivation. Explain why this project is of interest. Place it into the context of the research discussed in class (or from recommended & outside readings). All projects should address at least one of the discourse phenomena discussed in class, or in the bibliography. So, don't just tell us how users will benefit from the application; also tell us how you will learn about discourse phenomena from building the application, and how users will benefit from the computer being able to deal with the discourse phenomenon/a.

Potential Problems. Discuss potential problems that might arise in development and usability of your system. How will you address them?

Evaluation.  This is key. How will you know if you have succeeded? Present ideas for how to evaluate the success of the system.

A Note on Scope. Keep the scope of the project in mind: you will only have five weeks to build the system and you are welcome to use any toolkits, off-the-shelf technologies, and other shortcuts we have discussed or that you find yourself. Just like in the real world, you need to think about the limitations imposed by time and the available technologies. These should constrain your proposal accordingly. Therefore, we don't expect you to build a sophisticated system. We do expect you to demonstrate your understanding of some of the theoretical and design concepts discussed.  An excellent simple system will score better than an incomplete complicated system.

Due for this Homework: Send/print a 3 page write-up that addresses all of these sections, including the feedback we sent you, and the discussions your group had subsequently.  Also send, 24 hours before class, 1 single slide about your project updates that your team will stand up and present to the class.

 

Assignment 6: Presentation and Write-up of Project

Dialogue System

Of course, big, complex systems don't necessarily equate to better results (or marks). While the amount of time you spent on the project is important, it's more important that you demonstrate understanding of the concepts and present your work in a clear way. (Creativity is good too!)

Hand in:

1. A video of the system being used
2. Instructions on how to run and use the system and the URL or other information about how we can use the system, if that's possible
3. The transcript of an actual interaction with the system
4. Final Paper
5. Presentation Materials (powerpoint, video, etc.)

• Motivation and background literature including:
• Related Work.
Discuss a minimum of four papers, and how they relate to your project. Choose two from the discourse theory side and two from the computational modeling side; These can be papers discussed in class, papers from the recommended reading list, or outside reading.
• Hypotheses, and how they were derived
• Technical description
• Description of how system runs
• Evaluation.
Report on your evaluation study.
• Post Mortem. Describe the outcome of your project. What worked and what didn't? Discuss the problems and successes that you encountered in every step of the project, from design, to execution to evaluation. Discuss which issues had to do with the discourse phenomena that you depended on, and which derived from purely technical issues.
• Limitations? Did you feel that the toolkit imposed limitations on your system? Explain. Other limitations?
• Draw Inferences to the Real World. Think about how the available technology might reflect real-world limitations in research and commercial applications. What would you have liked to do if the technology were available?