More coursework: 1 - A | B | C | D | E | F | G | H | I - J | K - L | M | N - O | P - S | T | U - Y

Natural language processing

There have been high hopes for Natural Language Processing. Natural Language Processing, also known

simply as NLP, is part of the broader field of Artificial Intelligence, the effort towards making machines think.

Computers may appear intelligent as they crunch numbers and process information with blazing speed. In truth,

computers are nothing but dumb slaves who only understand on or off and are limited to exact instructions. But

since the invention of the computer, scientists have been attempting to make computers not only appear intelligent

but be intelligent. A truly intelligent computer would not be limited to rigid computer language commands, but

instead be able to process and understand the English language. This is the concept behind Natural Language


The phases a message would go through during NLP would consist of message, syntax, semantics,

pragmatics, and intended meaning. (M. A. Fischer, 1987) Syntax is the grammatical structure. Semantics is the

literal meaning. Pragmatics is world knowledge, knowledge of the context, and a model of the sender. When

syntax, semantics, and pragmatics are applied, accurate Natural Language Processing will exist.

Alan Turing predicted of NLP in 1950 (Daniel Crevier, 1994, page 9):

"I believe that in about fifty years' time it will be possible to program computers .... to

make them play the imitation game so well that an average interrogator will not have more than

70 per cent chance of making the right identification after five minutes of questioning."

But in 1950, the current computer technology was limited. Because of these limitations, NLP programs of

that day focused on exploiting the strengths the computers did have. For example, a program called SYNTHEX

tried to determine the meaning of sentences by looking up each word in its encyclopedia. Another early approach

was Noam Chomsky's at MIT. He believed that language could be analyzed without any reference to semantics or

pragmatics, just by simply looking at the syntax. Both of these techniques did not work. Scientists realized that

their Artificial Intelligence programs did not think like people do and since people are much more intelligent than

those programs they decided to make their programs think more closely like a person would. So in the late 1950s,

scientists shifted from trying to exploit the capabilities of computers to trying to emulate the human brain. (Daniel

Crevier, 1994)

Ross Quillian at Carnegie Mellon wanted to try to program the associative aspects of human memory to

create better NLP programs. (Daniel Crevier, 1994) Quillian's idea was to determine the meaning of a word by the

words around it. For example, look at these sentences:

After the strike, the president sent him away.

After the strike, the umpire sent him away.

Even though these sentences are the same except for one word, they have very different meaning because of the

meaning of the word "strike". Quillian said the meaning of strike should be determined by looking at the subject.

In the first sentence, the word "president" makes the word "strike" mean labor dispute. In the second sentence, the

word "umpire" makes the word "strike" mean that a batter has swung at a baseball and missed.

In 1958, Joseph Weizenbaum had a different approach to Artificial Intelligence, which he discusses in this

quote (Daniel Crevier, 1994, page 133):

"Around 1958, I published my first paper, in the commercial magazine Datamation. I

had written a program that could play a game called "five in a row." It's like ticktacktoe, except

you need rows of five exes or noughts to win. It's also played on an unbounded board; ordinary

coordinate will do. The program used a ridiculously simple strategy with no look ahead, but it

could beat anyone who played at the same naive level. Since most people had never played the

game before, that included just about everybody. Significantly, the paper was entitled: "How to

Make a Computer Appear Intelligent" with appear emphasized. In a way, that was a forerunner

to my later ELIZA, to establish my status as a charlatan or con man. But the other side of the

coin was that I freely started it. The idea was to create the powerful illusion that the computer

was intelligent. I went to considerable trouble in the paper to explain that there wasn't much

behind the scenes, that the machine wasn't thinking. I explained the strategy well enough that

anybody could write that program, which is the same thing I did with ELIZA."

ELIZA was a program written by Joe Weizenbaum which communicated to its user while impersonating a

psychotherapist. Weizenbaum wrote the program to demonstrate the tricky alternatives to having programs look at

syntax, semantics, or pragmatics. One of ELIZA's tricks was mirroring sentences. Another trick was to pick a

sentence from earlier in the dialogue and return it attached to a leading phrase at random intervals Also, ELIZA

would watch for a list of key words, transform it in some way, and return it attached to a leading sentence. These

tricks worked well under the context of a psychiatrist who encourages patients to talk about their problems and

answers their questions with other questions. However, these same tricks do not work well in other situations.

In 1970, William Wood, AI researcher at Bolt, Beranek, and Newman, described an NLP method called

Augmented Transition Network. (Daniel Crevier, 1994) Their idea was to look at the case of the word: agent

(instigator of an event), instrument (stimulus or immediate physical cause of an event), and experiencer (undergoes

effect of the action). To tell the case, Filmore put restrictions on the cases such as an agent had to be animate. For

example, in "The heat is baking the cake", cake is inanimate and therefor the experiencer. Heat would be the

instrument. An ATN could mix syntax rules with semantic props such as knowing a cake is inanimate. This

worked out better than any other NLP technique to date. ATNs are still used in most modern NLPs.

Roger Schank, Stanford researcher (Daniel Crevier, 1994, page 167):

"Our aim was to write programs that would concentrate on crucial differences in

meaning, not on issues of grammatical structure .... We used whatever grammatical rules were

necessary in our quest to extract meanings from sentences but, to our surprise, little grammar

proved to be relevant for translating sentences into a system of conceptual representations."

Schank reduced all verbs to 11 basic acts. Some of them are ATRANS (to transfer an abstract

relationship), PTRANS (to transfer the physical location of an object), PROPEL (to apply physical force to an

object), MOVE (for its owner to move a body part), MTRANS (to transfer mental information), and MBUILD (to

build new information out of old information). Schank called these basic acts semantic primitives. When his

program saw in a sentence words usually relating to the transfer of possession (such as give, buy, sell, donate, etc.)

it would search for the normal props of ATRANS: the object being transferred, its receiver and original owner, the

means of transfer, and so on If the program didn't find these props, it would try another possible meaning of the

verb. After successfully determining the meaning of the verb, the program would make inferences associated with

the semantic primitive. For example, an ATRANS rule might be that if someone gets something they want, they

may be happy about it and may use it. (Daniel Crevier, 1994)

Schank implemented his idea of conceptual dependency in a program called MARGIE (memory, analysis,

response generation in English.) MARGIE was a program that analyzed English sentences, turned them into

semantic representations, and generated inferences from them. Take for example: "John went to a restaurant. He

ordered a hamburger. It was cold when the waitress brought it. He left her a very small tip." MARGIE didn't work.

Schank and his colleagues found that "any single sentence lends itself to so many plausible inferences that it was

impossible to isolate those pertinent to the next sentence." For example, from "It was cold when the waitress

brought it" MARGIE might say "The hamburger's temperature was between 75 and 90 degrees, The waitress

brought the hamburger on a plate, She put the plate on a table, etc." The inference that cold food makes people

unhappy would be so far down the line that it wouldn't be looked at and as a result MARGIE wouldn't have

understood the story well enough to answer the question, "Why did John leave a small tip?" While MARGIE

applied syntax and semantics well, it forgot about pragmatics. To solve this problem, Schank moved to Yale and

teamed up with Professor of Psychology Robert Abelson. They realized that most of our everyday activities are

linked together in chains which they called "scripts." (Daniel Crevier, 1994)

In 1975, SAM (Script Applied Mechanism), written by Richard Cullingford, used an automobile accident

script to make sense out of newspaper reports of them. SAM built internal representations of the articles using

semantic primitives. SAM was the first working natural language processing program. SAM successfully went

from message to intended meaning because it successfully implemented the steps in-between - syntax, semantics,

and pragmatics.

Despite the success of SAM, Schank said "real understanding requires the ability to establish connections

between pieces of information for which no prescribed set of rules, or scripts, exist." (Daniel Crevier, 1994, page

167) So Robert Wilensky created PAM (Plan Applier Mechanism). PAM interpreted stories by linking sentences

together through a character's goals and plans.

Here is an example of PAM (Daniel Crevier, 1994):

John wanted money. He got a gun and walked into a liquor store. He told the owner he wanted some

money. The owner gave John the money and John left.

In the process of understanding the story, PAM put itself in the shoes of the participants. From John's

point of view:

I needed to get some dough. So I got myself this gun, and I walked down to the liquor store. I told the

shopkeeper that if he didn't let me have the money then I would shoot him. So he handed it over. Then I left.

From the store owner's point of view:

I was minding the store when a man entered. He threatened me with a gun and demanded all the cash

receipts. Well, I didn't want to get hurt so I gave him the money. Then he escaped.

A new idea from MIT is to grab bits and parts of speech and ask for more details from the user to

understand what it didn't before and to understand better what it did before (G. McWilliams, 1993).

In IBM's current NLP programs, instead of having rules for determining context and meaning, the

program determines its own rules from the relationships between words in its input. For example, the program

could add a new definition to the word "bad" once it realized that it is slang for "incredible." IBM also uses

statistical probability to determine the meaning of a word. IBM's NLP programs also use a sentence-charting

technique. For example, charting the sentence "The boy has left" and storing the boy as a noun phrase allows the

computer to see that the subject of a following sentence beginning with "He" as "the boy." (G. McWilliams, 1993)

In the 1950s, Noam Chomsky believed that NLP consisted only of syntax. With MARGIE, Roger Schank

added semantics. By 1975, Robert Wilensky's PAM could handle pragmatics, too. And as Joe Weizenbaum did

with ELIZA in 1958, over 35 years later IBM is adding tricks to its NLP programs. Natural Language Processing

has had many successes - and many failures. How well can a computer understand us?

Source: Essay UK -

About this resource

This coursework was submitted to us by a student in order to help you with your studies.

Search our content:

  • Download this page
  • Print this page
  • Search again

  • Word count:

    This page has approximately words.



    If you use part of this page in your own work, you need to provide a citation, as follows:

    Essay UK, Natural Language Processing. Available from: <> [06-06-20].

    More information:

    If you are the original author of this content and no longer wish to have it published on our website then please click on the link below to request removal: