ITI220 – Inverted Indexing Exercise, An inverted index

ITI220: Inverted Indexing Exercise List Student Names in Group (and circle your name): An inverted index, which is a common data structure in IR systems, is a list of all the index terms in a collection with pointers to the documents in which each term occurs, as well as the frequency that a term appears in the document and its location in the document. Specifically, an inverted index consists of:

• A collection of postings lists – one associated with each unique term in the collection. • Each posting list consists of a number of individual postings. • Each posting holds a unique document identifier (docno) and the frequency (count) of the term in a given document.

Let’s look at an example starting with the following famous lines from Shakespeare’s Merchant of Venice

Now, let’s treat each line (above) as if it were a “document.” The completed inverted index would look something like this:

Let’s interpret one of the terms – if – in the inverted index:

The number directly after the term is its document frequency or df for short. The df specifies the number of documents that contain this term. Since if appears in all four documents, its df is 4. Although the df can be easily reconstructed by counting the number of postings, it is often explicitly stored in the inverted index. The postings list contains a number of postings, each of which is a (docno, tf) tuple. The docno is simply a unique identifier for the document (one through four, in this case). The tf, which stands for term frequency, is the number of times the term appears in the document. The term if appears once in every document. Typically, postings are sorted by ascending docno (as shown above).

Exercise #3 – Instructions

1. Get in small groups of 2-4. The instructor will distribute the “In-Class Activity: Exercise #3” handout to each student. Put the names of your group at the top of the handout and circle your name. Each student will complete the activity handout.

2. As a group, select ONE of the document collections (see below – circle your choice and the group should select the same document collection) from Shakespeare’s famous quotes to create an inverted index. Each line is ~ to a document. (Quotations are from famous William Shakespeare Quotes, http://www.brainyquote.com/quotes/authors/w/william_shakespeare.html)

3. Working together as a group, each member will complete “In-Class Activity Exercise #3” and write an inverted index (on the handout) that would be built for the selected document collection. Refer to the “If you prick us” inverted index example for guidance in creating your group’s inverted index. Note: The terms are to be listed in alphabetical order.

4. As a group, discuss and write your response to the following question (in your handout): what did you learn about indexing as it pertains to information retrieval from this exercise?

5. Submit your group’s inverted index (and the selected Shakespeare Document Collection) to the instructor at the end of class. This exercise will be graded based on the “Information Retrieval Exercises Rubric.”

Shakespeare Document Collection #1 1. doubt thou the stars are fire 2. doubt that the sun doth move 3. doubt truth to be a liar, but 4. never doubt I love Shakespeare Document Collection #2 1. a fool thinks himself to be wise 2. but a wise man knows himself to be a fool Shakespeare Document Collection #3 1. we know what we are, but 2. know not what we may be Shakespeare Document Collection #4 1. when a father gives to his son, both laugh 2. when a son gives to his father, both cry Shakespeare Document Collection #5 1. we know what we are 2. but know not what we may be Shakespeare Document Collection #6 1. some are born great 2. some achieve greatness 3. and some have greatness thrust upon them Group’s Response to the Following Question: What did you learn about indexing as it pertains to information retrieval from this exercise?

Explanation & Answer

Our website has a team of professional writers who can help you write any of your homework. They will write your papers from scratch. We also have a team of editors just to make sure all papers are of HIGH QUALITY & PLAGIARISM FREE. To make an Order you only need to click Order Now and we will direct you to our Order Page at Litessays. Then fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.