Find the relevance of documents

Due: 12 Sep 2012

Educational objectives: Review C++ material that you have learned, such as implementing classes, using templates, performing I/O, and command line arguments. Use of STL classes (which you will learn in Lecture 4) and algorithms. Use of makefiles. Use of a debugger. Use of optimization flags with g++.

Statement of work: Develop a software that uses STL classes and algorithms, such as (linked) list or vector, to implement a simple document retrieval program, as described below. You must use at least one templated STL container and use its iterator.

Deliverables: Turn in a makefile and all header (*.h) and cpp (*.cpp) files that are needed to build your software. Turn in your development log too, which should be a plain ASCII text file called LOG.txt in your project directory. You will submit all of these as described in www.cs.fsu.edu/~asriniva/courses/DS12/HWinstructions.html.

Requirements:

• Create a subdirectory called proj1.
• You will need to have a makefile in this directory. In addition, all the header and cpp files needed to build your software must be present here, as well as the LOG.txt file.
• You should implement appropriate classes for the software. Your code should be well designed and object oriented.
• Your software's main task is as follows. A user will give it queries consisting of a set of words. For each query, the software should give the most relevant documents that contain all words in that query.
• The software is run by the user on the command line, as follows:
• Retrieve Filename-List, where Filename-List is a list of file names of zero or more ASCII text documents.
• The software first analyzes each file given on the command line. Details of the analysis are explained later. The software then waits for a series of user actions, and responds to each user action as described below.

  Possible user actions and required software response:

• a Filename: Analyze the ASCII text file Filename. If this file has already been analyzed previously, then replace the results of the previous analysis with the current one. If the file does not exist, then output File Filename does not exist to standard output (not to standard error).
• q n Word-List: n is a positive integer, and Word-List is a list of words (this is defined later) separated by one or more blanks. The software returns a list of the n most relevant documents containing all the words in Word-List. If fewer than n documents match the query, then all the matching documents are returned. Each line of the output will first give the name of a document, followed by a blank, followed by the document's relevance, which is a floating point number defined later. The documents are output in decreasing order of relevance (that is, most relevant document first). If no document matches, then output No matching document to standard output.
• x: Quit the program.
• Output Invalid comand to standard output for any other command.
• Analysis of a document. In this process, the software will identify the set of words in the document. Each word will be given a floating point weight. The weight of a word is the ratio of the frequency of its occurrence to the total number of words in the document. You should store the weights of each word, of each document analyzed, in a suitable container.

  A word is defined as a sequences of adjacent characters in the input file, separated by any of the following delimiters: whitespace (blanks, tabs, and newlines) or any of the following ! ( ) - : ; " , . ? /. (A delimiter cannot be a part of any word.) Note that abandon and Abandon are different words, as are car and car's.

• Relevance of a document. A document is relevant to a query only if it contains each word in the query. If a document contains each word in the query, then its relevance is defined as the sum of the weights, in this document, of all the words in the query.

Sample file and executable: A sample executable and some example files are available in the ~cop4530/fall12/solutions/proj1 directory on linprog. The first person to find an error in our executable will get a bonus point!

Bonus points:

You will get 3 bonus points if your code is faster than our sample executable on some large tests which we will announce after the submission deadline. (Your code should also be correct.)

You may get up to 5 additional points for significant extra work, such as implementing more features (for example, determining that different forms of the same word, such as serve, served, serving, and serves are equivalent) or providing a GUI interface. Please obtain written permission from me prior to doing this. If you wish to get bonus points, then please submit your work as usual, but send an email to the John Nguyen. John will schedule a meeting with you, and you can demonstrate the special features of your software then.

Copyright: Ashok Srinivasan, Florida State University.

---

Last modified: 29 Aug 2012