class: center, middle # CS-3110: Formal Languages and Automata ## Context-Free Grammars ### Chapter 4.1 --- # Regular and Non-Regular languages * Recall: We used regular expressions to define regular languages * Regular languages have certain restrictions, such as a lack of "counting" * For many applications, in particular in programming, we **need** "counting" in our language * One of the most common instances is matching parenthesis --- # Arithmetic Expressions * Take the language of (infix) arithmetic expressions over one-letter variables using the operators +, *, and parenthesis * This is an important part of many programming languages (but significantly easier to discuss than, say, the entirety of the Java language) * A valid expression follows certain rules: operators connect two operands, parenthesis are balanced and properly nested --- # Arithmetic Expressions: Examples Some sample expressions: * `a + b` * `(a + b)*c - (d*(c+b+a))` * `(a+b)*(b+c)` * `(((((a + b)))))` * `(((((a + a)*b + a)*b + a)*b + a)*b + a)*b` We can also combine any of these expressions by connecting them with any of the operators, expand them, and generally produce infinitely many different expressions --- # Arithmetic Expressions What is the general structure of an expression? An expression is either: * Atomic: A variable * The sum, difference, product or quotient of two sub-expressions * A sub-expression put in parenthesis --- class: small # Example `(a+b)*(b+c)` is: * The product of: - The sum of: + The atomic expression/variable `a` + The atomic expression/variable `b` - The sum of: + The atomic expression/variable `b` + The atomic expression/variable `c` --- # Example `(a+b)*(b+c)` as a tree: <center> <img src="/CS3110/assets/img/parsetree.png" width="50%"/> </center> --- # Abstract Syntax Tree * This "tree" that we just constructed is an abstraction of the syntax (structure) of our expression * We can construct such a tree for any valid expression * Given a tree, we can also write down the corresponding expression * A compiler can use such a tree to generate machine code; an interpreter can evaluate the expression with it * Now we just need a way to **define** "valid" trees --- class: medium # "Valid" Trees * How could we define how a tree is "constructed"? * For each operator we can define which types of children it can have * The children can then be constructed according to their own rules * In other words, each operator only needs to worry about the type of its children, not how they are constructed * The root can then be one particular "type" of node --- class: medium # Grammars A formal (context-free) grammar is a four-tuple: $$ (V,\Sigma, P, S) $$ * `\(V\)` is a set of **non-terminal symbols** ("variables" in our grammar) * `\(\Sigma\)` is a set of **terminal symbols** (the actual "symbols" in our produced words, i.e. the alphabet) * `\(P\)` is a set of **production rules** of the form `\(A \rightarrow w\)` with `\(A \in V\)` and `\(w \in (V\cup \Sigma)^\ast\)` * `\(S\)` is one of the non-terminal symbols from V, and called the **start symbol** --- # What does this mean? * The non-terminal symbols are placeholder nodes * The terminal-symbols are the actual nodes in our final tree * The rules define how to expand placeholder nodes * The start symbol defines a particular placeholder node to start with * Note: This view of the grammar as a tree is a bit of a simplification; we will discuss parse trees after the break --- # Rewriting System A concrete example, for our expression language: $$ V = \\{E, X\\} \\\\ \Sigma = \\{a,b,\ldots, z, +, *, (, )\\} $$ $$ P = \\{ E \rightarrow X, E \rightarrow E + E\\\\ E \rightarrow E * E, E \rightarrow ( E )\\\\ X \rightarrow a, X \rightarrow b, \ldots, X \rightarrow z \\} $$ $$ S = E $$ --- # Just the Rules $$ E \rightarrow X\\\\ E \rightarrow E + E\\\\ E \rightarrow E * E\\\\ E \rightarrow ( E ) \\\\ X \rightarrow a \\\\ X \rightarrow b \\\\ \vdots\\\\ X \rightarrow z $$ We start with the start symbol E. To **generate** a word, we can choose an arbitrary rule with E on the left-hand size, and replace the E with the right hand side of that rule. --- # As a Tree * We start with E as the root of our tree * We then **expand** it by adding children depending on the rule we apply * Once all of our nodes only contain **terminal symbols** we are done * The generated tree represents a valid expression (or, generally, a valid word in the language defined by the grammar) --- # As a Rewriting System * As mentioned, the tree-idea is a bit of a simplification * What we actually do is we start with the symbol "E", and then apply rules, **rewriting** what we have: $$ \begin{aligned} E \Rightarrow& E \ast E \\\\ \Rightarrow& ( E ) \ast E \\\\ \Rightarrow& ( E + E ) \ast E \\\\ \Rightarrow& ( X + E ) \ast E \\\\ \Rightarrow& ( a + E ) \ast E \\\\ \vdots \end{aligned} $$ --- # Complete Rewriting * If we just apply one rule at a time, this process takes a while * But we can apply multiple rules at the same time, too: $$ \begin{aligned} E \Rightarrow^\ast& E \ast E \\\\ \Rightarrow^\ast& ( E ) \ast ( E ) \\\\ \Rightarrow^\ast& ( E + E ) \ast ( E + E ) \\\\ \Rightarrow^\ast& ( X + X ) \ast ( X + X ) \\\\ \Rightarrow^\ast& ( a + b ) \ast ( b + c ) \end{aligned} $$ --- class: mmedium # Extended Rewriting * The `\(\Rightarrow\)` operation "rewrites" our current string by applying a single rule * In automata we had a similar idea, the "transition function", and then we defined an "extended transition function" * Likewise, we define `\(\Rightarrow^\ast\)` as the extended rewriting operation * However, because we are not "consuming" anything, it is instead defined as "zero or more repetitions of `\(\Rightarrow\)`" * `\(\Phi \Rightarrow^\ast \Psi\)` is true if `\(\Psi\)` can be generated by zero or more applications of `\(\Rightarrow\)` to `\(\Phi\)` --- # The Language Defined by a Grammar * Now we have a grammar `\(G\)` * The language defined by this grammar is: $$ L(G) = \\{w \in \Sigma^\ast | S \Rightarrow^\ast w\\} $$ Note: Since `w` can only consist of actual letters, this means we apply rules until we only have terminal symbols left. **Any** word that can be produced this way is a word in our language. --- class: small # Context-Free Grammars * The grammars we have defined are called **context-free** grammars * At the end of the semester we will briefly discuss more general grammars, which allow more than just a single single non-terminal symbol on the left-hand side of a rule * With this more general framework, one could replace a **sequence** of symbols (i.e. a symbol with some "context") with a rule * Context-Free Grammars are nice, because the languages they define can be recognized relatively easily * Context-Sensitive, or even more advanced grammars, are harder (or even impossible) to recognize in the general case --- class: mmedium # A Word on Rigor * Technically a grammar is a four-tuple, as described above * The "interesting" part are the rules * We often shorten writing grammars to *only* listing the rules * The canonical start symbol is then just called "S", and upper-case letters are (usually) used for non-terminal symbols * When in doubt (or when asked on homework or the exam), it is not wrong to explicitly state the set of non-terminal symbols, terminal symbols and name the start symbol! --- # Another Example * Before the exam, we showed that `\(\{a^n \cdot b^n | n \in \mathbb{N}\}\)` is not regular * But maybe we can use a context-free grammar to define it? -- $$ S \rightarrow a S b \\\\ S \rightarrow \varepsilon $$ --- # One more example $$ S \rightarrow aSa\\\\ S \rightarrow bSb\\\\ S \rightarrow cSc \\\\ S \rightarrow M \\\\ M \rightarrow aa \\\\ M \rightarrow cc \\\\ M \rightarrow bb $$ Which of these words is an element of the language defined by this grammar? abc, aa, abba, cbc, cbaccabc, bbbb --- # Shorthand $$ S \rightarrow aSa\\\\ S \rightarrow bSb\\\\ S \rightarrow cSc \\\\ S \rightarrow M \\\\ M \rightarrow aa \\\\ M \rightarrow cc \\\\ M \rightarrow bb $$ If we have many rules with the same left hand side, we often write them with an "or": $$ S \rightarrow aSa\\: |\\: bSb\\: |\\: cSc\\: |\\: M\\\\ M \rightarrow aa \\:|\\: cc\\: |\\: bb $$ --- # Let's see what ChatGPT thinks about this ... <div class="imessage"> <p class="from-me">Write a context-free grammar for the language over the alphabet {a,b,c} of words that contain more <code>a</code>s than <code>b</code>s.</p> <p class="from-them no-tail"> Here's a context-free grammar (CFG) for the language over the alphabet {a,b,c} of words that contain more "a"s than "b"s:<br/> <code> S -> aSa | aSb | aSc | a | c | ε </code> <br/> Explanation:<br/> - S is the start symbol.<br/> - aSa represents adding an "a" on both sides of the string.<br/> - aSb represents adding an "a" on the left and a "b" on the right.<br/> - aSc represents adding an "a" on the left and a "c" on the right.<br/> - a, c, and ε are terminal symbols representing an "a", "c", and empty string respectively.<br/><br/> This grammar ensures that any word generated by it will contain more "a"s than "b"s, as the only way to add "b"s is in the form of aSb, and there are more "a"s on the left side than "b"s on the right. </p> </div> --- # Let's see what ChatGPT thinks about this ... <div class="imessage"> <p class="from-me">Write a context-free grammar for the language over the alphabet {a,b,c} of words that contain more *a*s than *b*s.</p> <p class="from-them no-tail"> Here's a context-free grammar (CFG) for the language over the alphabet {a,b,c} of words that contain more "a"s than "b"s:<br/> <code> S -> aSa | aSb | aSc | a | c | ε </code> <br/> Explanation:<br/> - S is the start symbol.<br/> - aSa represents adding an "a" on both sides of the string.<br/> - aSb represents adding an "a" on the left and a "b" on the right.<br/> - aSc represents adding an "a" on the left and a "c" on the right.<br/> - a, c, and ε are terminal symbols representing an "a", "c", and empty string respectively.<br/><br/> This grammar ensures that any word generated by it will contain more "a"s than "b"s, as <b>the only way to add "b"s is in the form of aSb, and there are more "a"s on the left side than "b"s on the right</b>. </p> </div> --- # Almost ... <center> <img src="/pageassets/memes/soclose.jpeg" width="60%"/> </center> What's wrong with ChatGPT's solution? -- <code> S -> aSb | abS | bSa | baS | Sc | Sa | cS | aS | a </code> --- # Regular Expressions? * We have now shown that we can do something with grammars that we could not do with regular expressions * But can we do everything we **can** do with Regular Expressions with grammars, too? * Yes! Our grammars are strictly more powerful than Regular Expressions * Let's look at how our Regular Expression operations would look like as a Grammar --- # Regular Expressions to Grammars * `\(ab \)` becomes `\(S \rightarrow ab\)` * `\(a|b\)` becomes $$ S \rightarrow a\\\\ S \rightarrow b $$ * `\(a^\ast\)` becomes $$ S \rightarrow aS\\\\ S \rightarrow \varepsilon $$ --- # Parsing * As you may have noticed, the process of determining whether a word is in a language is tricky * We will see a way to do this "mechanically" after the break * We will also see a relatively small extension to our automata that will allow them to do the parsing for us!