Announcements:

• The mass collaboration annotation experiment is over. Thank you all for your participation!
• Midterm solution is available here
• Assignment #4 is available now. Click here.
• A note on the EM algorithm is now available. Click here.
• The midterm review list is now available. Click here.
• A list of sample project topics is now available on the Web (click on here).
• Assignment 3 is now available
• A note on KL-divergence retrieval formula is available here
• The due date for assignment 2 is now extended to Oct. 10, Friday, 2pm.
• The final examination has been scheduled on Dec. 16, Tuesday, 1:30-4:30pm.
• Typing your written answers for an assignment now entitles you to get an extra credit of 5 points! (We don't want to eliminate the possibility of handwritten answers, but a computer printout is *much* more preferred; there were some difficulties in reading some of your handwritten report for assignment one.)
• Guidelines on finishing the optional course project are now available.
• The deadline for assignment one has been extended to Sept. 19.
• The description of assignment one has been updated to correct an error in the perl command.
• Midterm is now scheduled to be on Oct. 22, Wednesday.
• Assignment one is now available.
• Office hour change on Fridays; it's now 9:15am-10:30am.
• Guidance on readings is now available.
• Please read all the information on this web site. Also, visit CSIL homepage and the Fall 2003 information.

Recent years have seen an explosive growth of the volume of online textual information (e.g., web pages, email, news articles, office documents, and scientific literature). It is a significant challenge to effectively manage such textual information and make it useful to people. For example, How do we search for interesting information in large collections of text? How do we automatically categorize a document according to a hierarchy of subject categories? Can we automatically discover the major complaints about a product from a large set of customer email messages to a company? These are only a few of many interesting and challenging questions that we can ask.

Unlike structured information, which is typically managed with a relational database, textual information is unstructured and poses special challenges due to the difficulty in precisely understanding natural language and users' information needs. In this course, we will introduce a variety of techniques for managing textual information, including algorithms for retrieval, filtering, clustering, and categorization of textual information. The course emphasizes basic principles and pratically useful algorithms. Topics to be covered include text analysis, retrieval models (e.g., Boolean, vector space, probabilistic), text categorization, text filtering, clustering, retrieval system design and implementation, and applications to web information management.

The course is lecture-based, and has a midterm and final examination. There will also be regular assignments, which often involve implementation of an algorithm and/or experimentation with real text data.