CS598-CXZ Advanced Topics in Information Retrieval (Spring 2005)

Course Project

Project proposals (click to see the proposals in the class Wiki place)

Introduction

The course project is to give you hands-on experience on doing research on some IR problem of your choice. You will be working on a small research topic, and writing a report about it. Team work is encouraged (just as in real life), but not required.

General steps

1. Pick a topic

You can either pick from a list of suggested topics , or choose your own topic. In general, a topic can fall into one or more of the following three categories:

Evaluation: Empirically evaluate one or more existing retrieval algorithms in order to better understand these methods, and possibly further improve them.
Original study: Propose a new algorithm or formula, and compare it with a baseline algorithm on a standard test set.
Application: Develop an IR application using an existing retrieval method.

When picking a topic, try to ask yourself the following questions:

Is the topic addressing an important problem (or an important need, in the case of building an application) ? Would any one care about it, if you solve the problem, or build the application?
Do you have any idea at all about how to solve the problem, or build the application? If not, can you reformulate the problem to make it easier?

A good topic is one that you like to work on, one that people care about, that you have some idea about it, and that is small enough for you to finish by the end of the semester.

2. Form a team

You are encouraged to work with other student as a team. This not only would give your some experience on working with others, but also would allow you to work on a larger (presumably more interesting) topic. You can use the class mailing list or newsgroup to communicate with each other to form teams.

3. Read related work

Once you have chosen a topic, you will need to read some papers to know about what people have already done on the topic, so that you can be sure your idea extends , rather than duplicates, the existing work, if any.

4. Write a proposal

If your project involves testing a hypothesis or original research, your proposal should state explicitly all the following:

What is your research question? Clearly define the research problem/question.
Why is this an interesting question to ask and why would we care about the answer to this question or a solution to the problem?
Has any existing research work tried to answer the same or a similar question, and if so, what is still unknown?
How do you plan to work out the answer to the question. (At the porposal stage, you are only expected to have a sketch of your methods .)
How would you evaluate your solution. That is, how do you plan to demonstrate that your solution/answer is good or is reasonable.
A rough timeline to show when you expect to finish what. List a couple of milestones.

If your project is to build a software tool, then your proposal should state explicitly all the following:

What is exactly the function of your tool? That is, what will it do?
Why would we need such a tool and who would you expect to use it?
Have people already built such a tool or a similar tool, and if so, how yours is different from or better than other tools. Do we already know how to build such a tool? What is the challenge?
How do you plan to build it?
How will you demonstrate the usefulness of your tool?
A rough timeline to show when you expect to finish what. List a couple of milestones.

VERY IMPORTANT: Please estimate the amount of work you propose to do and make sure that you can actually finish the project by the end of this semester. It is okay to propose an ambitious project, but you will need to specify clearly what you are commited to actually complete for the course project.

Please post your proposal on the Wiki of the course before Feb. 22, 2005. Your proposal will be peer-reviewed and the instructor will grade your proposal based on the peer reviews.

5. Work on the project

You should reuse any existing tools as much as possible. There are many tools available on the Internet. See the resources page for some useful pointers.

Discuss any problems or issues with your teamates. Discuss them with other students and the instructor. If you need special support (e.g., more disk space on your account), please let the instructor know.

Consider documenting your work regularly. This way, you will already have a lot of things written down by the end of the semester.

You are all granted permissions to do cgi programming on the CSIL student web server. Detailed information is described in the following excerpt from a message:

The web server is www-students.cs.uiuc.edu . In each student's CSIL
account, they should create a directory ~YOURNETID/public_html/cgi-bin . This
is where their cgi scripts should be stored. The scripts will then be called
with an address like this,
http://www-students.cs.uiuc.edu/~YOURNETID/cgi-bin/YOURSCRIPT

6. Project progress report (presentation schedule)

Sometime in the middle of the project, you will be asked to present a project progress report to the class. You do NOT need to turn in any written progress report. Each project is allocated a 15 minute slot for presenting and discussing the progress of the project. It is entirely up to the project group who will actually do the presentation. It is fine if one of the project group members presents the work while the others help answer any questions, and it is also fine if the group decides to have multiple people to speak. You should plan to finish your presentation within 10-12 minutes to leave enough time for questions and feedback from the audience. The following is a list of the elements you should include in your presentation, along with a suggested breakdown of the time. These are meant to be guidelines, though; you should make adjustment according to the actual contents of your presentation.

[2 min] Concise introduction to the problem you work on
[3 min] Briefly describe the main idea of your project (expected contribution of your project)
[3 min] Briefly explain what you have accomplished so far and any lessons you've learned so far
[2 min] Briefly discuss the remaining tasks
[1 min] List any issues you want to discuss with the audience and any issues you want your classmates to give you feedback on

Each project will be graded based on the actuall progress you have made as well as the quality of your presentation. Please see the grading criteria for more information about how these two factors are weighted. The quality of the presentation will be judged mainly based on five factors: (1) how informative the contents of your slides are; 2) organization of the presentation; (3) clarity of the presentation; (4) the effectiveness of using time; (5) how well you (as a group) answer questions. In general, all the students in a project group will get the same grade for project progress report.

7. Present your project

At the end of the semester, each team is expected to make a poster presentation of the project. Details will be posted later.

8. Write a project report (due May 8, 2005, Sunday)

You should write your report as if you were writing a short conference paper. You can think of it as an expanded written description of your poster presentation; you may also think of your poster presentation as a summary of the report. Thus the same general guideline applies, i.e., you should (1) explain your problem clearly; (2) provide sufficient motivation for your work and explain how your work is connected with the existing/previous work; (3) explain your methods with sufficient details; (4) discuss the research results; (5) summarize your work, draw conclusions if possible, and discuss how you think the work can be further improved/extended. There is no strict length requirement. You may target at anywhere between 6 pages (font 10, single column) and 10 pages (font 11, single column) without counting any necessary appendices. Actually, given the same amount of essential information, the shorter the better; of course, you will have to judge what counts as "essential information". A good report is not just a straightforward description of what you did (Such a paper would probably never be accepted by a good conference); it should demonstrate your research contributions very clearly and convincingly. Thus it is important that you think very clearly about what are the major points you want to make and include arguments and empirical evidence that support your points. For example, whenever you make a choice of some method or design an algorithm in a particular way, always justify it, at least, try to explain why you have made this choice. Also, you may want to summarize or plot your experiment results in a particular way rather than some other ways, because the "particular way" would support/illustrate your point better. Always keep in mind what exactly you expect your readers to learn from your report, including both positive and negtive findings.

The project report should be handed in by email, and is due on May 8, Sunday. I will post all the reports on the web when I receive them.