summary |
shortlog | log |
commit |
commitdiff |
tree
first ⋅ prev ⋅ next
Nick Downing [Sun, 27 Jan 2019 23:05:32 +0000 (10:05 +1100)]
Remove obsolete work.py module, and accidental generation of b.xml diagnostic
Nick Downing [Sun, 27 Jan 2019 11:30:07 +0000 (22:30 +1100)]
Remove obsolete __repr__() and repr_serialize() on AST elements, change spacing
Nick Downing [Sat, 26 Jan 2019 23:29:20 +0000 (10:29 +1100)]
Remove obsolete way of initializing AST elements with serialized (string) data
Nick Downing [Mon, 21 Jan 2019 07:28:51 +0000 (18:28 +1100)]
Make piyacc.py standalone
Nick Downing [Mon, 21 Jan 2019 06:19:37 +0000 (17:19 +1100)]
Get direct AST generation working properly, does not exactly match bootstrap_bison.git and bootstrap/ parser because of a small whitespace issue with %code { }
Nick Downing [Mon, 21 Jan 2019 01:45:08 +0000 (12:45 +1100)]
Implement a stack of position markers to delay the insertion of an opening tag until just before the corresponding closing tag, update scan-gram.l in root also
Nick Downing [Mon, 21 Jan 2019 01:24:45 +0000 (12:24 +1100)]
Add element-based Python scanner/parser in root of repository which builds AST directly (but for now, only scan-code.l builds AST directly, then converts back to text), and n.sh test script to check same result as bootstrap/ scanner/parser
Nick Downing [Mon, 21 Jan 2019 00:31:51 +0000 (11:31 +1100)]
Tidy scan-code.l, fix bug in scan-gram.l not setting yylval for element groups
Nick Downing [Sun, 20 Jan 2019 09:58:05 +0000 (20:58 +1100)]
Allow value and stack references in action text of (?{...}...) element groups, add tests_ast/cal_py.y tests which use action text (but not value or stack references, need another test later), update yytoken to yychar in generate_py.py
Nick Downing [Sun, 20 Jan 2019 04:03:10 +0000 (15:03 +1100)]
Improve error handling in Python parser
Nick Downing [Sat, 19 Jan 2019 10:49:19 +0000 (21:49 +1100)]
Add bootstrap/n.sh test script, fix ../../bootstrap_bison.git/src/parse-gram.y
Nick Downing [Fri, 18 Jan 2019 01:58:01 +0000 (12:58 +1100)]
Implement re-scanning of action text to insert ValueReference, StackReference etc, processes ../tests/cal.y and ../../boostrap_flex.git/src/parse.y correctly
Nick Downing [Thu, 17 Jan 2019 23:12:16 +0000 (10:12 +1100)]
Add Python version of bootstrap_bison.git parser (adds markup to *.y file)
Nick Downing [Mon, 14 Jan 2019 14:57:25 +0000 (01:57 +1100)]
Make non-element scanner ignore take_space_left which is always True for empty production, change yytoken (which was private to flex) to yychar, add YYEMPTY
Nick Downing [Sat, 12 Jan 2019 13:09:58 +0000 (00:09 +1100)]
Make --element parser capture final space before EOF
Nick Downing [Fri, 26 Oct 2018 01:13:50 +0000 (12:13 +1100)]
Add LICENSE and README.md, add license information to every source file
Nick Downing [Thu, 27 Sep 2018 05:15:34 +0000 (15:15 +1000)]
Implement %space, which controls how element groups capture inter-token space
Nick Downing [Wed, 26 Sep 2018 14:04:47 +0000 (00:04 +1000)]
Implement element groups with support from piyacc (not via user code)
Nick Downing [Wed, 26 Sep 2018 12:10:56 +0000 (22:10 +1000)]
Add element groups into the AST and post processing, currently they're ignored
Nick Downing [Wed, 26 Sep 2018 07:44:47 +0000 (17:44 +1000)]
Make tests_ast directory use element groups and store mantissa/fraction in AST
Nick Downing [Wed, 26 Sep 2018 07:38:38 +0000 (17:38 +1000)]
Update skel_py_element.py and tests_ast directory to pilex.git commit
dd0c364
Nick Downing [Thu, 13 Sep 2018 11:15:19 +0000 (21:15 +1000)]
Extend calculator example to build an AST and re-calculate expressions from it
Nick Downing [Wed, 12 Sep 2018 13:02:02 +0000 (23:02 +1000)]
First cut at automatic AST generation with piyacc.py --element switch
Nick Downing [Tue, 11 Sep 2018 10:36:17 +0000 (20:36 +1000)]
Add interactive parser ability, tidy up the Python test and skeleton slightly
Nick Downing [Tue, 11 Sep 2018 05:37:37 +0000 (15:37 +1000)]
Rename pyacc to piyacc, rename PYACC object and prefix to AST
Nick Downing [Mon, 10 Sep 2018 04:33:08 +0000 (14:33 +1000)]
Add --python switch, skel/skel_py.py template, and tests/cal_py.y test parser
Nick Downing [Fri, 10 Aug 2018 12:43:26 +0000 (22:43 +1000)]
Rename skeleton to skel_bison.c, include skel_bison.c in repository instead of requiring bootstrap_bison, fix skeleton patches that were not applying properly
Nick Downing [Wed, 8 Aug 2018 06:08:05 +0000 (16:08 +1000)]
Move BisonLR1DFA generation into LR1DFA.to_bison_lr1dfa() consistent with other generators, and skeleton filling code from bison_lr1dfa.py to generate_bison.py
Nick Downing [Fri, 3 Aug 2018 07:08:03 +0000 (17:08 +1000)]
Put PYACC.Text inside PYACC.BracedCode excluding the braces, except for the action text where this is too difficult (so we'll work around it in ast.py for now)
Nick Downing [Mon, 30 Jul 2018 00:37:52 +0000 (10:37 +1000)]
Hack to fix $$ from yyval to yylval, @$ from yyloc to yylloc in initial action
Nick Downing [Sun, 29 Jul 2018 22:35:30 +0000 (08:35 +1000)]
Add %initial-action, and add some forgotten location handling code in skeleton
Nick Downing [Sun, 29 Jul 2018 11:54:58 +0000 (21:54 +1000)]
Egregious hacks, and fix a genuine bug in the #ifdef YYLTYPE_IS_DECLARED stuff
Nick Downing [Sun, 29 Jul 2018 10:32:57 +0000 (20:32 +1000)]
Implement %define api.prefix and %name-prefix (nearly the same but not quite)
Nick Downing [Sun, 29 Jul 2018 09:46:40 +0000 (19:46 +1000)]
Remove %<flag> stuff in favour of explicit Section1.Debug|Locations|PureParser, implement %debug (AKA api.parse_trace) and %error-verbose (parse.error verbose)
Nick Downing [Sun, 29 Jul 2018 09:04:40 +0000 (19:04 +1000)]
Implement %locations (inserts code that ensures YYLTYPE_IS_DECLARED is defined)
Nick Downing [Sun, 29 Jul 2018 07:22:02 +0000 (17:22 +1000)]
Fix missing #defines for tokens bug in y.tab.h, generate YYSTYPE not just union
Nick Downing [Sun, 29 Jul 2018 06:46:40 +0000 (16:46 +1000)]
Implement %define api.pure (sets #define YYPURE to 0=false, 1=true, 2=full)
Nick Downing [Sun, 29 Jul 2018 06:19:06 +0000 (16:19 +1000)]
Minor tweaks for building bison (now works, but don't have %define api.XXX yet)
Nick Downing [Sun, 29 Jul 2018 02:19:24 +0000 (12:19 +1000)]
Fix bug with reference offset for mid-rule actions, ensure all symbols typed
Nick Downing [Sun, 29 Jul 2018 01:54:48 +0000 (11:54 +1000)]
Get rid of set_code_props() in favour of the ordinary post_process(), fix union
Nick Downing [Sun, 29 Jul 2018 01:37:16 +0000 (11:37 +1000)]
Put a PYACC.SymbolRef inside PYACC.Production.SymbolRef to do the heavy lifting, and eliminate symbol attribute from the inner one in favour of a direct return
Nick Downing [Sun, 29 Jul 2018 00:09:17 +0000 (10:09 +1000)]
Combine PYACC.TerminalRef and PYACC.NonterminalRef into PYACC.SymbolRef (relying on information passed in from the surrounding context to set the symbol type)
Nick Downing [Sat, 28 Jul 2018 23:53:27 +0000 (09:53 +1000)]
Get rid of TaggedSymbols, just put tags/symbols inside Section1.Type or similar
Nick Downing [Sat, 28 Jul 2018 23:39:29 +0000 (09:39 +1000)]
Make *_to_symbol contain symbol index rather than positive terminal index or negative nonterminal index, get rid of pyacc.terminals[] and pyacc.nonterminals[]
Nick Downing [Sat, 28 Jul 2018 00:30:29 +0000 (10:30 +1000)]
Implement pyacc.symbols instead of pyacc.(non)terminals, combine PYACC.Terminal and PYACC.Nonterminal into PYACC.Symbol with a _type field to say whether terminal or nonterminal, make the external interface with bison_lr1dfa.py use these
Nick Downing [Thu, 26 Jul 2018 14:36:32 +0000 (00:36 +1000)]
First cut at making the semantic analysis accept what it needs to build bison
Nick Downing [Tue, 24 Jul 2018 09:43:25 +0000 (19:43 +1000)]
Fix --defines command line option and fix escaping of "'\"'", for building flex
Nick Downing [Sun, 22 Jul 2018 13:59:04 +0000 (23:59 +1000)]
Remove split out versions of regex.py, nfa.py, dfa.py, grammar.py since they are now safely in regex.git commit
bb6ca2e with fixes, minor renaming in bison_lr1dfa.py for the sake of consistency with flex_dfa.py in plex2.git commit
88d529f
Nick Downing [Fri, 20 Jul 2018 12:01:56 +0000 (22:01 +1000)]
Implement %union/<tag>, add https://github.com/gautamkmr/Compiler-Lab- examples
Nick Downing [Fri, 20 Jul 2018 07:21:31 +0000 (17:21 +1000)]
Implement midrule actions, and fix bug which suppressed actions not using 12660/
Nick Downing [Fri, 20 Jul 2018 04:55:54 +0000 (14:55 +1000)]
Improve how production lengths are tracked and subtracted from StackReferences
Nick Downing [Fri, 20 Jul 2018 04:33:37 +0000 (14:33 +1000)]
Remove PYACC.Section base class and make PYACC.Section1Or2 the direct parent
Nick Downing [Thu, 19 Jul 2018 13:27:01 +0000 (23:27 +1000)]
Rationalize PYACC.Text so it occurs only once inside containers like PYACC.Char, and so that embedded references like PYACC.Escape now become PYACC.Text.Escape and occur inside PYACC.Text instead of being interspersed with PYACC.Text nodes
Nick Downing [Thu, 19 Jul 2018 12:38:35 +0000 (22:38 +1000)]
Fix serialization so that nodes (e.g. PYACC.Production) can be parented within the main document and also be referenced (e.g. from the list PYACC.productions)
Nick Downing [Thu, 19 Jul 2018 00:43:39 +0000 (10:43 +1000)]
Change 'utf-8' encoding back to 'unicode' since xml.etree uses this to decide whether the stream is text or binary, change (first|start)_nonterminal and (last|precedence)_terminal usage again so that we can see it in the serialized output
Nick Downing [Thu, 19 Jul 2018 00:18:26 +0000 (10:18 +1000)]
Split out PYACC.Symbol into PYACC.Terminal and PYACC.Nonterminal with no precedence on the latter, rename PYACC.Production.Symbol to PYACC.Production.SymbolRef
Nick Downing [Thu, 19 Jul 2018 00:05:43 +0000 (10:05 +1000)]
Tidy up handling of associativities by adding an extra table which is indexed by the precedence, instead of using (precedence, associativity) pairs everywhere
Nick Downing [Wed, 18 Jul 2018 23:43:31 +0000 (09:43 +1000)]
Get precedences working in the YACC/Bison way where both terminals and productions have precedences, done this by removing the priority field in lr1.productions and instead adding a new field lr1.precedences containing (terminal_breaks, terminal_prec, nonterminal_breaks, nonterminal_prec) similar to action/goto tables
Nick Downing [Wed, 18 Jul 2018 13:18:59 +0000 (23:18 +1000)]
Improve how first_nonterminal and last_terminal are handled (may be overridden)
Nick Downing [Wed, 18 Jul 2018 12:45:37 +0000 (22:45 +1000)]
Split out PYACC.Section1Or2.Symbol into PYACC.TerminalRef, PYACC.NonterminalRef
Nick Downing [Wed, 18 Jul 2018 12:19:09 +0000 (22:19 +1000)]
Partially revert commit
8494a18, since the LR1DFA itself does not need the compressed token numbering by virtue of how it uses bisect maps (to support unicode)
Nick Downing [Wed, 18 Jul 2018 05:31:55 +0000 (15:31 +1000)]
Change group_bounds per-production field of LR1 and LR1DFA to generic ref_data, comment out the parser code which treats it as group_bounds for now, and use ref_data field to hold PYACC.BracedCode to execute when the production is reduced
Nick Downing [Tue, 17 Jul 2018 13:31:06 +0000 (23:31 +1000)]
Avoid use of the Grammar and Grammar.Production objects, instead merge Grammar logic directly into the PYACC object and rename Section2.Rules.RHS to Production
Nick Downing [Tue, 17 Jul 2018 06:55:02 +0000 (16:55 +1000)]
Rework token numbering so that the LR1DFA is generated using the internal numbering for the terminals, and characters/user token numbers are supported properly
Nick Downing [Tue, 17 Jul 2018 02:14:50 +0000 (12:14 +1000)]
Rework how tag/symbol lists from the parser are handled, collect precedence and associativity data per symbol, create explicit terminal symbols for characters (to hold the precedence and associativity), currently leaves gaps in numbering
Nick Downing [Wed, 11 Jul 2018 07:14:26 +0000 (17:14 +1000)]
Change lookahead_item_shift() to lookahead_item_set_action() to reflect that it returns a set of reductions as well as a description of the state after shiting
Nick Downing [Sun, 8 Jul 2018 05:39:58 +0000 (15:39 +1000)]
Split out regex.py module into regexes, NFAs, DFAs, grammars, LR1s and LR1DFAs
Nick Downing [Sun, 8 Jul 2018 05:19:46 +0000 (15:19 +1000)]
Decouple Regex and LR1 by defining a new Grammar.Production.Symbol class, simplifies the handling of separate terminal and nonterminals sets, but means that symbols in a production can no longer be grouped (Nick's advanced feature) for now
Nick Downing [Fri, 6 Jul 2018 12:22:17 +0000 (22:22 +1000)]
Fix default reduction offset by 2 issue, parser now avoids useless lookahead
Nick Downing [Fri, 6 Jul 2018 06:38:59 +0000 (16:38 +1000)]
Fix EOF action and other things to get parser working, still lookahead problem
Nick Downing [Thu, 5 Jul 2018 23:07:02 +0000 (09:07 +1000)]
Much improved version with proper handling for tokens, actions and so on
Nick Downing [Thu, 5 Jul 2018 12:10:27 +0000 (22:10 +1000)]
First cut at table generation and packing, almost OK but doesn't compile yet
Nick Downing [Wed, 4 Jul 2018 13:10:10 +0000 (23:10 +1000)]
Store nonterminal symbols in a separate array so that the position in this array gives a unique numeric index per nonterminal symbol, as opposed to the character_set which is non-unique as it refers to a set of indices in productions array
Nick Downing [Wed, 4 Jul 2018 10:03:44 +0000 (20:03 +1000)]
Renaming a number of variables in regex.py for consistency (no other changes)
Nick Downing [Tue, 3 Jul 2018 12:32:52 +0000 (22:32 +1000)]
Implement semantic analysis pass to build a regex.Grammar object, then LR1, etc
Nick Downing [Mon, 2 Jul 2018 12:26:24 +0000 (22:26 +1000)]
First cut, loads and deserializes tree, few post_process() routines implemented