Procedure:
Collect a sample of naturally occurring spoken discourse in which grounding phenomena occur. "Naturally-occurring" can be broadly construed to include radio or talk shows, children's spontaneous play, radio or TV news items, spontaneous or scripted storytelling, classroom interactions, task-oriented conversations, classroom lectures, etc. By "collect" we mean make video recordings (unless absolutely impossible to do and audio recording is your only option, in which case you should explain this choice of data). You should collect a minimum of 15 minutes, and then (1) transcribe and (2) code at least 5 continuous minutes (usually the middle of the discourse is the most natural). By "transcribe" we mean you should make a record on paper of what you saw/heard --a good enough record so that when we read the transcript, we know what went on (read the Schiffrin appendices and Ochs before doing this. You will probably want to use one of her methods, or you will have to justify why you didn't). You may use a pre-existent coding scheme (one of the ones we have discussed in class, or from an article that interests you), or you may make one up.
Discussion:
1. The Transcript: The point is to push you to think about what discourse is and what makes it hard to model discourse in a computational system. You may want to have an interactive system in mind when you choose your sample. Think about how a computer could replace a participant in the discourse. Another point is to think about what makes a sufficient record of discourse: how do you turn a speech event into an on-paper transcript? What parameters need to be transcribed (the words, the pronunciation of words, the intonation, the facial expression, the gestures, fidgeting, pauses, etc.)?
2. The Coding: The point is to push you to think about what phenomena you are looking for, and how to isolate them in the stream of language. How do you think through what phenomenon you are interested in / what actual events make up the phenomenon, and how to find the boundaries of the instances of those events.
What you must turn in:
We ask you to turn in to us (1) the typed transcript, (2) the coded transcript, and (3) a ONE PAGE discussion of the points listed above. That is, minimally, discuss the issue of what makes an adequate transcription, and what phenomenon you chose to code, and how you chose/made up a coding scheme, and what challenges a computer might have in interacting in the discourse that you have collected.

(Note: Feedback will be given the following week, after which you will have 4 weeks to build the system and perform your evaluation)

Overview. For your final project you will be developing an application or system that accounts for the grounding phenomena we have been discussing in class. Possible topics could include a novel configuration of a video-mediated communication system that you use to further describe how various forms of visual feedback influence grounding, or a dialogue-based navigation system (e.g., a Garmin GPS system) that makes use of previous conversations to inform future direction giving behaviors, or a conversational robot that adapts its speech based on rules derived from the cognitive aging literature.

Describe Domain and Application. Choosing a good domain is crucial for innovative research and systems design. The domain should serve both to provide an interesting foundation for developing a research application and to constrain the task for which your application is designed.

Motivation. Explain why this project is of interest. Place it into the context of the research discussed in class (or from recommended & outside readings). All projects should address the grounding phenomena we have discussed in class. So, don't just tell us how users will benefit from a particular application; also tell us how you will learn about grounding or discourse phenomena as a result of building and evaluating an application.

Potential Problems. Discuss potential problems that might arise in development and usability of your system. Present initial ideas for how to evaluate the success of the system.

A Note on Scope. Keep the scope of the project in mind: you will only have four weeks to build a system and evaluate some grounding behaviors. You may use any of the tools that we have discussed or used in class, and the use of other technologies are allowed and encouraged. Just like in the real world, you need to think about the limitations imposed by time and the available technologies. These should constrain your proposal accordingly. Therefore, we don't expect you to build a sophisticated system. We do expect you to demonstrate your understanding of the theoretical concepts we have discussed and we expect you to ground your design in this understanding.

Final Project Deliverables. We will talk more about this, but in the end we expect a paper describing your system and study and a final presentation of your application and relevant study.

Due for this Homework: Send a 2 page write-up (no more than 3pages) by email to Brooke (b-foucault AT northwestern DOT com) and the instructors (dgergle AT / justine AT / whorton AT northwestern DOT edu).

Procedure: The goal of this homework is to get you to think about how you might use eye-tracking data to develop a richer understanding of conversational grounding processes. You will make use of previously collected eye-gaze data captured during actual conversational exchanges to propose a research question and way of addressing the question (although you will not have to fully answer the question, since that would take much longer!). To successfully complete this assignment, you will need to devise a method for using these data to answer a particular research question with respect to grounding. To do this, there are a number of major steps you’ll need to perform:

1. Devise a research question that you can investigate using a combination of spoken discourse and eye-tracking data.
2. Choose a data set from the choices we provide you, based on the kind of question you want to address. You can find the raw eye-tracker videos and their short descriptions on blackboard in the "assignments" folder.
3. Once you have chosen your data set, watch it carefully and devise an annotation scheme that will allow you to partition the data in such a way that allows you to capture how the eye-gaze patterns and fixations proceed over time. Think carefully about the form of the representation you’ll need in order to successfully answer your research question. Be sure to do the following:
1. Describe what are referred to as “regions of interest” in the video (e.g., quadrants or areas of interest where a participant might be looking)
2. Think about how you will make a determination as to whether a person is actually “looking” (i.e., fixating) within a particular region or not.
3. Develop a frame-by-frame method for assessing when a participant is looking at a defined region. Be specific about what it takes to do this objectively and accurately!
4. Devise a method for integrating your eye-tracking data with the spoken discourse.
5. Use these data to answer your research question.

What You Must Turn In: Hand in a 2-page summary that describes: (1) Your research question, choice of data, and specific information on how you expected to answer your question making use of the eye-tracking data.  (2) The process you used to transform the raw data into a usable form. Be specific about decisions you made and where you ran into challenges. Detail how and why you made particular decisions in transforming the data. The point here is similar to the coding homework where we want you to think hard about both the phenomena you are looking for, and how you can isolate them in a multi-stream environment.  Make sure to always tie your answers to the research question you propose. (3) Discuss any problems that arose in your data collection and partitioning of the eye-tracking data. Describe how you might overcome some of these problems in the future. (4) Provide a brief description of the ways you think this information, or your findings, could be used in the development of a particular computational application. (5) Include as an appendix an excerpt of the data in the form you used for your analysis. For example, you might show a graph of the utterances with interspersed gaze patterns.

Due Date:
November 20th (during class on *Tuesday* of this week)

Final Project: Description and Deliverables

Your final project may take one of a variety of different forms, and the deliverables will partially depend on the form of your final project.  We have described a number of options for the final project, that can be summarized like this:

Kinds of final projects

• Based on data you collect, build a formal computational model for some specific aspect of grounding behavior, and demonstrate how your model fits a new set of data.
• Based on data you collect, build a system that can communicate with real people and engage in a particular type of grounding behavior, and then evaluate that system with two or three users (no need for statistical significance).
• Find an existing system that is meant to engage in grounding behavior with people and, based on concepts from class, make changes to that system, and then evaluate your changes with a couple of users.
• Build an end-to-end system that demonstrates a phenomenon we have talked about in class and conduct a heuristic evaluation that demonstrates how it works (or shows why it doesn't work).
• Build an automatic/computational tool that aids in the in the analysis of a specific aspect of grounding and evaluate your tool using existing data or data that you collect.

Based on those descriptions, here are some things that most of the projects will need to include:

System

1. The files that make up your system
2. Instructions on how to run and use the system
3. If yours is the kind of project without actual participants evaluating; then the transcript of an actual interaction with the system
4. Documentation: If you write any code, it must be documented

Evaluation
A protocol including description of participants, task, coding and analysis
A transcript of at least one participant’s interaction with the system
Numeric results of the evaluation 

Write up

• Related Work. Why did you choose to do the project you did in terms of previous literature. Discuss a minimum of 4-6 papers, and how they relate to your project. It’s wise to choose 2-3 from the discourse theory side and 2-3 from the computational modeling side. These can be papers discussed in class, papers from the recommended reading list, or outside reading.
• System. What exactly does your system do? Why does it do what it does?
• Evaluation. Report on your evaluation study.
• Post Mortem. Describe the outcome of your project. What worked and what didn't? Discuss the problems and successes that you encountered at every step of the project, from design, to execution to evaluation. Discuss which issues had to do with the discourse phenomena that you depended on, and which derived from purely technical issues.
• Limitations? What were the limitations of your system. How might you overcome them in a subsequent iteration?
• Draw Inferences to the Real World. Think about how the available technology might reflect real-world limitations in research and commercial applications. What would you have liked to do if the technology were available?

In your write-up remember that big, complex systems don't necessarily equate to better results (or grades). While the amount of time you spent on the project is important, it's more important that you demonstrate understanding of the concepts and present your work in a clear way. (Creativity is good too :)

The final write-up should be under 20 pages, plus the System and Evaluation information.  Your group may turn in one-single write-up however in that case you must include a half-page statement on who did what, in order to adhere to Northwestern’s regulations.