Find the relevance of documents

Due: 20 Oct 2012

Educational objectives: Experience implementing a self-organizing linked-list and solving problems using the above class.

Statement of work: (i) Implement a linked list class that self-organizes as specified below and (ii) implement a document retrieval program, which is a modification of that in assignment 1, but using your self-organizing linked list to store the list of words in each document. You should not use an STL list in your implementation.

Deliverables: Turn in a makefile and all header (*.h) and cpp (*.cpp) files that are needed to build your software. Turn in your development log too, which should be a plain ASCII text file called LOG.txt in your project directory. You will submit all of these as described in www.cs.fsu.edu/~asriniva/courses/DS12/HWinstructions.html.

Requirements:

• Create a subdirectory called proj3.
• You will need to have a makefile in this directory. In addition, all the header and cpp files needed to build your software must be present here, as well as the LOG.txt file.
• You should implement appropriate classes for the software. Your code should be well designed and object oriented.
• Your software's main task is as follows. A user will give it queries consisting of a set of words. For each query, the software should give the most relevant documents that contain all words in that query.
• The software is run by the user on the command line, as follows:
• Retrieve Filename-List, where Filename-List is a list of file names of zero or more ASCII text documents.
• The software first analyzes each file given on the command line. Details of the analysis are explained later. The software then waits for a series of user actions, and responds to each user action as described below.

  Possible user actions and required software response:

• a Filename: Analyze the ASCII text file Filename. If this file has already been analyzed previously, then replace the results of the previous analysis with the current one. If the file does not exist, then output File Filename does not exist to standard output (not to standard error). This command reads each word in the file. Each time a new word is encountered, it is pushed to the back of your linked list (there is a distinct list for each document).
• q n Word-List: n is a positive integer and Word-List is a list of words (this is defined later) separated by one or more blanks. The software returns a list of the n most relevant documents containing all the words in Word-List. If fewer than n documents match the query, then all the matching documents are returned. Each line of the output will first give the name of a document, followed by a blank, followed by the document's relevance, which is a floating point number defined later. The documents are output in decreasing order of relevance (that is, most relevant document first). If no document matches, then output No matching document to standard output. This command also self-organizes the linked list of each document as follows. For each word in the word list that is found, that word is moved two steps toward the front of the list (unless it is already the first or second word, in which case it is placed in the front of the list).
• p n Filename: n is a positive integer and Filename is the name of a file that has previously been analyzed. If this file has not been analyzed previously, then output File Filename has not been analyzed to standard error. Otherwise, output the first n words in the linked list that contains all the words in Filename.
• x: Quit the program.
• Output Invalid command to standard output for any other command.
• Analysis of a document. In this process, the software will identify the set of words in the document. Each word will be given a floating point weight. The weight of a word is the ratio of the frequency of its occurrence to the total number of words in the document. You should store the weights of each word, of each document analyzed, in a suitable container.

  A word is defined as a sequences of adjacent characters in the input file, separated by any of the following delimiters: whitespace (blanks, tabs, and newlines) or any of the following ! ( ) - : ; " , . ? /. (A delimiter cannot be a part of any word.) Note that abandon and Abandon are different words, as are car and car's.

• Relevance of a document. A document is relevant to a query only if it contains each word in the query. If a document contains each word in the query, then its relevance is defined as the sum of the weights, in this document, of all the words in the query.

Sample file and executable: A sample executable and some example files will be available in the ~cop4530/fall12/solutions/proj3 directory on linprog later. The first person to find an error in our executable will get a bonus point!

Bonus points:

You will get 3 bonus points if your code is faster than our sample executable on some large tests which we will announce after the submission deadline. (Your code should also be correct.)

You may get up to 5 additional points for significant extra work, such as implementing more features (for example, determining that different forms of the same word, such as serve, served, serving, and serves are equivalent) or providing a GUI interface. Please obtain written permission from me prior to doing this. If you wish to get bonus points, then please submit your work as usual, but send an email to the John Nguyen. John will schedule a meeting with you, and you can demonstrate the special features of your software then.

Copyright: Ashok Srinivasan, Florida State University.

---

Last modified: 15 Oct 2012