テクスト語彙処理プログラムLEXの開発について : 概要と論理

LEX is a program package now being developed for various analyses of a text. LEX as a system has the following characteristics. First, it is intended for the use of researchers in social sciences and humanities who often deal with linguistic data but are not familiar with a computer-assisted data processing. Secondly, the system deals solely with characters and character strings, and not with numeric data. Thirdly, it is not a single-purpose system, but a general-purpose system, in that it is provided with a variety of ways of processing data. The basic unit of processing by LEX is called 'word', which is defined as a string of characters. In addition, LEX can also deal with substrings of 'words' and strings of 'words' (called lexical strings). Moreover, LEX can perform data-selection operations at the level of substring, 'word' and lexical string. These characteristics enable the user to obtain a rich variety of results. The user of LEX communicates with the system by means of what might be called the LEX language. Its grammar is very limited and simple, and, roughly speaking, can be described by a few context-sensitive rewriting-rules. The system itself consists of two components: one interpreting input LEX sentences, and the other performing data processings proper, such as compilation of frequency table, alphabetical list, cooccurrence table, concordance and so on.