Name: Syntactic structures
ISBN: 9780899250908

Syntactic structures

American linguist Paul Postal wrote in 1964 that most of the "syntactic conceptions prevalent in the United States" were "versions of the theory of phrase structure grammars in the sense of Chomsky". British linguist John Lyons wrote in 1966 that "no work has had a greater influence upon the current linguistic theory than Chomsky's Syntactic Structures." Prominent historian of linguistics R. H. Robins wrote in 1967 that the publication of Chomsky's "Syntactic Structures" was "probably the most radical and important change in direction in descriptive linguistics and in linguistic theory that has taken place in recent years". Another historian of linguistics Frederick Newmeyer considers "Syntactic Structures" "revolutionary" for two reasons. Firstly, it showed that a formal yet non-empiricist theory of language was possible and more importantly, it demonstrated this possibility in a practical sense by formally treating a fragment of English grammar. Secondly, it put syntax at the center of the theory of language. Syntax was recognized as the focal point of language production, in which a finite set of rules can produce an infinite number of sentences. As a result, morphology and phonology were relegated in importance. "Syntactic Structures" also initiated an interdisciplinary dialog between philosophers of language and linguists.

What Hacker News says

bborOct 2024

Well said! This is a great restatement of the core setup of the Chomskian “Generative Grammar” school, and I think it’s an undeniably productive one. I haven’t read this researchers full paper, but I would be sad (tho not shocked…) if it didn’t cite Chomsky up front. Beyond your specific point re:interfaces—which I recommend the OG Syntactic Structures for more commentary on—he’s been saying what she’s saying here for about half a century. He’s too humble/empirical to ever say it without qualifiers, but IMO the truth is clear when viewed holistically: language is a byproduct of hierarchical thought, not the progenitor.

This (awesome!) researcher would likely disagree with what I’ve just said based on this early reference:

  In the early 2000s I really was drawn to the hypothesis that maybe humans have some special machinery that is especially well suited for computing hierarchical structures.

…with the implication that they’re not, actually. But I think that’s an absurd overcorrection for anthropological bias — humans are uniquely capable of a whole host of tasks, and the gradation is clearly a qualitative one. No ape has ever asked a question, just like no plant has ever conceptualized a goal, and no rock has ever computed indirect reactions to stimuli.

thread

helwrJun 2010

Managing Gigabytes (Witten)

Information Retrieval (Manning)

Text Compression (Bell)

Natural Language Processing (Manning)

Natural Language Understanding (Allen)

Speech and Language Processing (Jurafsky)

The Text Mining Handbook (Sanger)

Statistical Machine Translation (Koehn)

Data-Intensive Text Processing with MapReduce (Lin)

Algorithms on strings (Gusfield)

Jewels of Stringology (Crochemore)

Regular Expressions (Friedl), also: http://swtch.com/~rsc/regexp/regexp1.html and automata theory (Hopcroft)

Practical Text Mining with Perl (Bilisoly)

Natural Language Processing with Python (Bird)

Computational Linguistics (Hausser)

Syntactic structures (Chomsky)

also check out these links: http://measuringmeasures.blogspot.com/2010/01/learning-about...

http://measuringmeasures.com/blog/2010/3/12/learning-about-m...

http://www.cs.technion.ac.il/~gabr/resources/resources.html

thread

PaulHouleOct 2026

Mainframe-era systems often used fixed-size or limited-size buffers and data structures, based on those 80-character punched cards, most addressing modes on the 360 add 12-bit immediate displacements to pointers held in registers [1] bytes and such.

It is easy to laugh but C programmers have demonstrated that they can't handle variable-sized buffers on the stack, malloc(), free() and all that.

Early versions of FORTRAN [2] didn't even have a stack, like you couldn't write recursive functions, the program would take parameters for array sizes when it boots and the memory layout would all be figured out at the beginning. When I was developing a display controller for persistence of vision displays for the AVR-8 it drove me nuts how the C compiler kept moving the stack pointer around all the time for no real reason when an assembly program could move data to and from statically allocated structures. [3]

FORTRAN and COBOL came out around the time Syntactic Structures did so of course there was not an established theory for how to parse grammars, no parser generators, etc. No Dragon Book, but the heroic techniques used to develop compilers in that age are legendary if not quite lost.

I wrote some FORTRAN programs in high school on a VAX, in a comparative programming languages class in college, converted a lot of FORTRAN algorithms into statically allocated Java in grad school in 1990 and in my postdoc I wrote a generator that wrote JNI bindings for LAPACK.

[1] ... e.g. the typical struct is < 4096 bytes

[2] before the FORTRAN 90 standard though you could set the /RECURSIVE flag on DEC FORTRAN

[3] we were done such a disservice by all those "it's so elegant to compute Fibonacci in a better programming language than BASIC" articles in Creative Computing because recursion is one of those things, like C's memory management, that most programmers can't really handle.

thread

PaulHouleDec 2025

The Chomsky Hierarchy isn't the right way to think about, in fact the Chomsky paradigm probably held back progress in language technology over the past 70 years.

In the Kuhnian sense, Syntactic Structures was the vanguard of a paradigm shift that let linguists do "normal science" in terms of positing a range of problems and writing papers about them. It was also useful in computer science for formally defining computer languages that are close enough to natural languages that people find them usable.

On the other hand, attempts to use the Chomsky theory to develop language technology failed miserably outside of very narrow domains and in the real world Markov models and conditional random fields often outperformed when the function was "understanding", "segmentation", etc.

From an engineering standpoint, the function that tells if a production is in the grammar that is central to the theory is not so interesting for language technology, I mean, if LLMs were -centric then an LLM would go on strike if you got the spelling or grammar wrong or correct you in a schoolmarmish William Safire way but rather it is more useful for LLMs to silently correct for obvious mistakes the same way that real language users do.

The other interesting thing is the mixing of syntax and semantics. For a long time I believed the "Language Instinct" idea of Pinker in which the ability to serialize and deserialize language is a peripheral that is attached to an animal and you need the rest of the animal and even the surrounding world to have "human" competence. Now LLMs come around and manage to show a lot of linguistic competence without the rest of the animal and boy we have egg on our face, and coming out of the shadow of the Chomsky theory, structuralism will rear it's head again. (e.g. "X is structured like a language" got hung up on the unspoken truth that "language is not structured like a language")

thread

nabla9Dec 2025

You saw name "Noam Chomsky" and that started a process in your mind that generated the standard spiel about Syntactic Structures.

Chomsky Hierarchy is his more fundamental work that joins computer science and linguistics. It was published in IRE Transactions on Information Theory. Chomsky, Noam (1956). "Three models for the description of language" https://chomsky.info/wp-content/uploads/195609-.pdf

Type-3 grammar ≡ finite-state automaton

Type-2 grammar ≡ Non-deterministic pushdown automaton

Type-1 grammar ≡ Linear-bounded non-deterministic Turing machine

Type-0 grammar ≡ Turing machine

ps. Chomsky was already aware of Finite State Automata and Turing Machines and understood that they match Type-3 and Type-0. Pushdown Automata was invented later, and connection between Type-1 grammar and Linear Bounded Automata was made few years later.

thread

mcswellMar 2022

I can't get to the article (I've read my two free articles for the month, and I'm a cheapskate), but I have a question. One sort of hybrid system would be of a kind that imposed constraints on the solution that the neural system could learn--not just preventing the wrong solutions, but channeling the network toward a good solution.

As I understand programs like BERT, they largely look at the words adjacent to a given word, without paying attention to any structure. Whereas in real human languages--any human language--there is a hierarchical structure. This has been known since 1957 (Chomsky's Syntactic Structures, although I suspect that context free or weakly context sensitive models are sufficient). Is there a way to force a neural net to construct hierarchical models? And of course the hierarchies are not different for every word--all nouns behave more or less the same, all intransitive verbs behave in a different way, etc. So the task of the neural net (or whatever learning system) is to discover the hierarchy for a given language.

Likewise, morphology is mostly suffixes and prefixes, with a handful of other possible structures (up to reduplication). And the affixes fit into a paradigmatic structure. The neural net should be primed to look for that kind of structure, and to back off to things like ablaut only if the affixal model doesn't work.

Is this a way that symbolic and neural approaches could play together? Let the symbolic channel and constrain the neural net.

thread