General-purpose macro processor

A general-purpose macro processor or general purpose preprocessor is a macro processor that is not tied to or integrated with a particular language or piece of software.

A macro processor is a program that copies a stream of text from one place to another, making a systematic set of replacements as it does so. Macro processors are often embedded in other programs, such as assemblers and compilers. Sometimes they are standalone programs that can be used to process any kind of text.

Macro processors have been used for language expansion (defining new language constructs that can be expressed in terms of existing language components), for systematic text replacements that require decision making, and for text reformatting (e.g. conditional extraction of material from an HTML file).

Examples of general purpose macro processors

Name	Year	Description
GPM	1960s	One of the earliest macro processors was GPM (the General Purpose Macrogenerator).[1] This was developed at the University of Cambridge, UK, in the mid 1960s, under the direction of Christopher Strachey.
ML/I	1960s	One particularly important general purpose macro processor was (and still is) ML/I (Macro Language One). This was developed as part of PhD research by a Cambridge postgraduate, Peter J. Brown. ML/I operates on a character stream, and requires no special format for its input, nor any special flag characters to introduce macros.
STAGE2	1960s	A contemporary of ML/I was STAGE2,[2] part of William Waite's Mobile Programming System.[3] This too is a general purpose macro processor, but it processes input a line at a time, matching each line against specified patterns; it is notable in that it is independent of character set, requiring only that the digits 0-9 are contiguous and in that order (a condition not met by some of the 6-bit and BCD character codes of the era).
M6	1960s	Early macro processor developed at AT&T Bell Laboratories by Douglas McIlroy, Robert Morris and Andrew Hall. It was influenced by GPM and TRAC. Implemented in FORTRAN IV,[4] it was ported to Version 2 Unix.
SNOBOL	1960s	SNOBOL is a string processing language which is capable of doing most of the pre-processing which can be done by a macro processor.
XPOP		XPOP was another attempt at a general macro processing language by Mark Halpern at IBM in the 1960s.
TTM	1968	TTM is a recursive, interpretive language designed primarily for string manipulation, text editing, macro definition and expansion, and other applications generally classified as systems programming. It was developed in 1968 by Steven Caine and E. Kent Gordon at the California Institute of Technology. It is derived, primarily, from GAP[5] and GPM.[1]
GMP	1970s	Another attempt was the GMP (General Macro Processor) developed in the mid-1970s by M Boule in the DLB/GC department of the CII Company along ideas from R.J. Chevance. Tested in association with the Bordeaux I University the first version ran the SIRIS8/IRIS80 System. It was ported to mini6 systems and was the main component involved in the system generation for this family of computers. The GMP processor used C2-Chomsky grammars to define the syntax of macros and used an imperative language to execute computations and proceed to macro expansion.
M4	1977	m4 was designed and written in C for Unix by Dennis Ritchie and converted to Ratfor by Brian Kernighan.[6]
ELENA		Software: Practice and Experience, Vol. 14, pp. 519–531, Jun. 1984
gema	1995	gema is a contextual macro processor based on pattern matching, written by David N. Gray. It replaces/enhances the concept of regular expressions by contexts. Contexts roughly corresponds to named sets of patterns. As a consequence, macros in gema closely resemble an EBNF description.[7]
GPP	1996	gpp is another general macro processor written by Denis Auroux. It resembles a C preprocessor, but has more general semantics and allows for customized syntax (for instance, TeX, XHTML, and Prolog-like scripts are definable).[8]
M5	1999	m5 is a general-purpose macro processor written by William A. Ward, Jr. Unlike many macroprocessors, m5 does not directly interpret its input. Instead it uses a two-pass approach in which the first pass translates the input to an awk program, and the second pass executes the awk program to produce the final output.
pyexpander	2011	pyexpander is a general-purpose macro processor based on the Python programming language. In addition to simple macro replacement it allows evaluation of arbitrary Python expressions and execution of python code.
Text Assembler	2014	Text Assembler is a general-purpose text/macro processor based on the JavaScript programming language. Beyond simple macro replacement, it allows evaluating arbitrary JavaScript expressions and executing JavaScript code. It can also load JSON data models for more complex data-driven text processing tasks.[9]
PP	2016	PP is a text preprocessor designed for Pandoc (and more generally Markdown and reStructuredText). PP implements: Macros, literate programming, GraphViz, PlantUML and ditaa diagrams, Bash, Cmd, PowerShell, Python and Haskell scripts.[10]
minimac		minimac is a minimalist general purpose macro processor. It operates as a character stream filter, recursively expanding macros as they are encountered. It is unusual for a macro processor in that it uses an explicit argument stack, and user functions are defined by concatenation (similar to the Forth language).[11]
aa_macro	2017	aa_macro is an open-source character-stream-based text processing language written in Python. Text is processed in a left-to-right, inside-to-outside manner. A selection of pre-defined built-in functions provide fundamental processing mechanisms that may be used directly or as elements of user-defined styles. The language is user extensible, and wtfm, an open-source web-based document preparation wrapper for the language, is available.[12][13]

See also

• Macro (computer science) – Rule for substituting a set input with a set output