EssayproSolutions

(Solved) : Given Text Corpus Develop Positional Index Process Phrase Proximity Queries Using Position Q26570194 . . .

Given a text corpus, develop a positional index. Process phraseand proximity queries using the positional index.

You will be provided with a text corpus, which will be comprisedof about 500 research papers published in Learning Analytics &Knowledge (LAK) conferences. The Society for Learning AnalyticsResearch (SoLAR) organizes LAK conferences. LAK conferences focuson research that explores the role and impact of analytics onteaching, learning, training and development. The corpus will beprovided in ASCII file format. Each document will come in aseparate physical file. You are responsible for normalizing thetext.

You have to follow these steps :

1- Normalize text – address punctuation characters,stemming/lemmatization, and lowercasing. Do not throw away stopwords.

2- Extract tokens and identify vocabulary for thedictionary.

3- Scan the corpus and build an inverted index.

4-  Scan the corpus and build a positional index usingthe inverted index. You may eliminate the inverted indexconstruction in the step above if you can figure out a way todirectly construct the positional index.

5-  Implement the algorithm for processingphrase/proximity queries. “I attached an image of this algorithm

6- Develop a simple interface for users to specifyphrase/proximity queries. Assume that the queries are limited tojust two terms. The interface can be as simple as prompting theuser for a phrase/proximity query (i.e., a text string). You mayalso read a phrase/proximity query through command linearguments.

7- Design test cases and execute them. Document executionresults.

POSITIONALINTERSECT(p1, p2, k) 2 3 answer ← while pl * NIL and P2 NIL do if docID(P1)=docID(p2) then I-〈 〉 stions(P) 6 7 8 pp2 positions(p2) while ppl NIL do while PP2 NIL do if Ipos(pp) pos(pp2)l Sk then ADD(l, pos(p/p2)) else if pos(pp2) > pos (ppi) 10 12 13 14 15 16 17 18 19 20 21 then break pp2 ← next(pp2) while I * 〈 〉 and IIO]-pos(pp1)| > k do DELETE(l[0) for each ps E I do ADD(answer, (doc! D (p1), pos(pp), ps〉) ppl ← next(ppl) p1 ← next (P1) P2-next (P2) else if docID(pi) < docID(p2) then pi next (pi) else p2 next(p2) 23 24 return answer Show transcribed image text

Expert Answer

Answer to Given Text Corpus Develop Positional Index Process Phrase Proximity Queries Using Position Q26570194 . . .