Word Extraction

Time limit1sMemory limit128 MB

Summary
Clean each input line by lowercasing it, joining or splitting words at punctuation by neighbor rules, then print the sorted unique words per line.
Level

Medium4 of 10

Topics
String, Sorting, Simulation
Solved
No attempts yet

Problem

I want a dictionary of the words my students actually use. Text from their assignments, forum posts, and emails goes into a program that pulls the words out.

To make it easy to tell whether a word has been seen before, the text is cleaned up first. Each line of text is processed in this order.

  1. Every upper case letter becomes lower case.
  2. A letter is one of a to z and a digit is one of 0 to 9. Every other character except the space is punctuation.
  3. If the character immediately before a punctuation mark and the character immediately after it are both letters, that mark is deleted and the two sides are joined. Otherwise the mark becomes a single space. The two neighbours are always read from the line before any processing, so a run of punctuation marks does not affect itself. A mark at the very start or the very end of a line has no character on one side, so it becomes a space.
  4. The resulting string is split on spaces to give the words.
  5. A word made only of digits is thrown away.
  6. The remaining words are sorted into alphabetical order and duplicates are removed. Comparison uses ASCII code values, so digits come before letters.

For instance haven't has a letter on both sides of the apostrophe and becomes havent, and Hartley-Jones. becomes hartleyjones. In top-10 the character after the hyphen is a digit, so it splits into top and 10, and 10 is thrown away for being all digits.

Input

Input consists of a number of lines, each line holding one piece of text to analyse. The input ends with a line containing the single character #, which is not text.

There is at least one line of text, and no line contains more than 250 characters. Every line is made of printable ASCII characters, that is codes 32 to 126.

Output

Print the qualifying words of each piece of text, one per line, in alphabetical order. One line of text gives one set of words, and two neighbouring sets are separated by a single blank line.

A line that yields no words still counts as a set. That set prints nothing while the blank lines separating it stay, so blank lines can appear one after another.

Examples2

  1. Example 1

    Input
    Can I please have the spec for the Programming 3 assignment? B.T.W. I haven't got the lecture notes either!
    Our guest speaker is Mr Hartley-Jones. He shouldn't need any introduction - you all met him last semester, didn't you?
    When you said "give me your best 5 examples", did you mean just from this year - or can we use any? I'd like to use one from2006.
    #
    
    Expected output
    assignment
    btw
    can
    either
    for
    got
    have
    havent
    i
    lecture
    notes
    please
    programming
    spec
    the
    
    all
    any
    didnt
    guest
    hartleyjones
    he
    him
    introduction
    is
    last
    met
    mr
    need
    our
    semester
    shouldnt
    speaker
    you
    
    any
    best
    can
    did
    examples
    from
    from2006
    give
    id
    just
    like
    me
    mean
    one
    or
    said
    this
    to
    use
    we
    when
    year
    you
    your
    
  2. Example 2

    Input
    hello world
    42 3.14 007
    Bye!
    #
    
    Expected output
    hello
    world
    
    
    bye