CS410 Text Information Systems (Spring 2015)

Course Project

Introduction

The course project is to give the students hands-on experience on developing some novel information retrieval and/or text mining tools. The project thus emphasizes applied research and "deliverables," meaning that the outcome of your project should be something tangible, typically some kind of prototype system that can be demonstrated, though theoretical research projects are also fine. Group work is strongly encouraged, but not required.

General steps

Grading criteria

Your project will be graded primarily based on the following weighting scheme:

The factors to be considered in grading include (1) the utility of the tool you develop: would anyone actually use it? (this is the most important factor); (2) the novelty of the tool: how different is it from the existing tools? (3) quality of presentation/writing; (4) relevance to the course.

1. Pick a topic

You can either pick from a list of sample topics provided by the instructor and TAs (will be available here) later, or choose your own topic. Your starting point should be the essays that you wrote for assignment #1.

You may find useful to take a look at some general advice on choosing a project topic here. Please keep in mind that the general goal is to develop some useful tools to help people manage and make use of text information. Leveraging existing resources is especially encouraged as it allows you to minimize the amount of work that you have to do and focus on developing truly novel functions rather than repeating what people have already done.

When picking a topic, try to ask yourself the following questions:

What is exactly the function of the tool that you would like to develop?
Is the envisioned tool really useful? Who will benefit from such a tool? Note that the usefulness of the tool is the most important grading factor, so make sure that you invest your effort on developing a really useful tool. It's fine if the tool is a very specialized tool, or only useful for a few people.
Does this kind of tools already exist? If similar tools exist, how is your tool different from them? Would people care about the difference? Why is this difference important?
Do you have some rough idea about how the target function might be achieved? (The instructor and TAs will help you answer this question.)
How do you plan to show people that the tool to be developed is indeed useful? (You could make a demo of its function and/or report some quantitative evaluation figures.)
What is the minimum goal to be achieved during this semester? (Try to drop everything non-essential and only keep the function that is truly novel.)

2. Form a team

You are encouraged to work with other students in a team. Teamwork not only gives your some experience on working with others, but also allows you to work on a larger (presumably more interesting) topic. Generally speaking, all the team members of a group will get the same grade provided all have contributed substantially to the project. In case there is evidence that a team member has only made superficial contribution to a project (I really hope this won't happen!), the particular team member's grade may be discounted. The project report must state clearly who did what. It is also fine if you choose not to work with others.

3. Check related work

While choosing a topic, you should also check to see whether the tool/function you would like to develop already exists. If so, you may want to figure out where exactly your novelty is and whether novelty leads to any benefit for a user. Your goal is to extend , rather than duplicate, the existing work. To minimize your effort, you should leverage existing systems, toolkits, and other useful resources as much as possible. The instructor and TAs can also help you check related work. Please feel free to discuss your plan with us at any time. Try to discuss with your peers as well.

4. Write a proposal (due date to be announced later)

You are required to write a one-page proposal before you actually go in depth on a topic and post your proposal on the class wiki by the due date.

To post your proposal, go to the Project Proposals page, and add your project to the page. Follow the same format as used by some existing project proposals there. Specifically, you need to create a new page for your project, where you would put your proposal. It is up to you how to design such a page; it could directly contain your proposal or have your proposal as an attachment to this new page. To do that, you simply need to edit the proposal wiki page and add a line like "# [Your project title]" to this page. After saving the page, you can click on the new entry you created, which would lead you to the new page that you created, and you can then edit that page.

In the proposal, you should address the following questions and include the names and email addresses of all the team members. (As long as these questions are addressed, the proposal does not have to be very long. A couple of sentences for each question would be sufficient.)

What is the function of the tool?
Who will benefit from such a tool?
Does this kind of tools already exist? If similar tools exist, how is your tool different from them? Would people care about the difference?
What existing resources can you use?
What techniques/algorithms will you use to develop the tool? (It's fine if you just mention some vague idea.)
How will you demonstrate the usefulness of your tool.
A very rough timeline to show when you expect to finish what. (The timeline doesn't have to be accurate.)

You will be asked to review a small number of other proposals. Guidelines are posted on the project proposal page .

5. Work on the project

You should reuse any existing tools as much as possible. See the resources page for some useful pointers.

Discuss any problems or issues with your teammates or classmates. Discuss them with the TAs and the instructor. If you need special support (e.g., more disk space on your account), please let the instructor know.

Consider documenting your work regularly. This way, you will already have a lot of things written down by the end of the semester.

6. Present the course project

At the end of the semester, each project team is expected to make a presentation of the project. The purpose of this presentation is: (1) Let you know about others' projects. (2) Give you some opportunity to practice presentation skills, which are very important for a successful career. (3) Obtain some feedback from others about your project. Every on-campus student is expected to attend this presentation unless you have obtained permission from the instructor in advance for not attending. The course project presentation will be given at 8am-11am, Thursday, May 14, 2015, in room 1320 DCL (our classroom). There will be multiple sessions (organized based on project topics) with short breaks in between. Everyone is expected to show up for all sessions.

If a team consists solely of online students, they can pre-record a short voiced Powerpoint presentation, which will be played at the presentation meeting. Please inform the instructor in advance if this is what you plan to do.

In general, the structure of your presentation should roughly follow your project proposal. So it should very briefly touch all the following aspects:

What is the function of your system? (or What research question do you address?)
Why is this function useful? (or why is it important/interesting to answer the question?)
Is this function already supported in any way by the current systems/tools (e.g., Google)? How novel is your project? (or to what extent has the existing work/literature answered this question?)
How did you implement this function? (or what method did you use to address your question?)
How well does your system work? (Show some sample results -- quantitative evaluation, screen shots,etc)
Any ideas for further improvement?

Your presentation will be graded mainly based on (1) the clarity of your slides and presentation, (2) whether your presentation has covered all the questions listed above, and (3) whether you can finish the presentation within the allocated time. Think about how you can best present your work so as to make it as easy as possible for your audience to understand your main messages. Try to be concise, to the point. Pictures, illustrations, and examples are generally more effective than text for explaining your project. Try to show screen shots and/or plots of your experimental results. If you are not familiar with PowerPoint, you can adapt this sample presentation.

7. Write a project report

Each team must submit a written project report by Wednesday, May 13, 2015. You should write your report as if you were writing a short conference paper. You should address the same questions as those you have addressed in the proposal, only with more details, especially regarding some of the challenges that you need to solve in developing the tool. You should also include some screenshots if applicable and any other evaluation results. Furthermore, it would be good to include a brief discussion of how your system/research work can be further extended.

There is no strict length requirement, but you may target at 2000~4000 words without counting any necessary appendices (this is about 4~6 pages with 10-point font or 6~8 pages with 11-point font). Feel free to use any format for your report.

If you have not written such a report before, you may want to take a look at the following sample research papers published in the World Wide Web conferences:

Of course, I would expect your reports to be much shorter and more concise, but you should try to write your reports in a similar way. You may also refer to this sample project report for an example of how to organize your project report. If you want to reuse this template for your report, you can download the Microsoft Word file here. If you want to use Latex to write your report, you can check out resources on Latex here.

The project report should be posted on the presentation schedule page in the same way as you've done for your progress report presentation slides. The deadline for posting your project reports is 11:59pm, May 13, 2015, Wednesday. There are instructions on this wiki page to tell you how exactly you can post your project report.

Each project team only needs to submit one report. However, if there are multiple members in the team, you must include, for each member, at least one sentence to describe what he/she did exactly for the project. .

Grading of a project report will be based on three factors: (1) [25%] clarity and completeness of the report itself (i.e., whether you have clearly described what you have done and addressed all the questions that you are supposed to address); (2) [50%] amount of work that you have done; and (3) [25%] the quality of your project as reflected in the importance of problem being addressed, the quality of solution, and the impact of your project. Since the report accounts for 40% of your overall grade for the project, it means that if you've devoted enough effort (i.e., getting 50% of the report grade), and also done well for the project proposal and project presentation, you should have at least 40*50%+30+30=80 points for your project grade, out of the total of 100 points. On the other hand, if there is no evidence showing that you have done substantial work for a group project and only made superficial contribution, then not only will you lose many of the 50% of the points for the amount of work you've done, but your grades in the parts of the project may be discounted as well. Thus it is very important that you make sure to make enough effort to contribute to your group project. Note that it is your responsibility to figure out how to contribute to your group project, so you will need to act proactively and in a timely manner if your group leader has not assigned a task to you. There will be no opportunity to make up for any task that you failed to accomplish. Everyone is expected to spend at least 20 hours to seriously work on your course project as a minimum, not including the time spent for preparing the presentation and writing the report. In general, all the members of a team will get the same grade for the project unless the report indicates that some member(s) only superficially participated in the project without doing much actual work; in that case, I will discount the grade. The instructor and TAs will provide feedback about your report later by email if we see any way to further improve your work.