logjson.git
2 years agoImplement LazyArray.splice(), implement unshift/shift/push/pop in terms of splice... master
Nick Downing [Sun, 13 Feb 2022 01:09:22 +0000 (12:09 +1100)]
Implement LazyArray.splice(), implement unshift/shift/push/pop in terms of splice (also, this fixes a bug with pop where the LazyArray.length was not updated)

2 years agoFix bug with not marking array or object dirty after deleting a property
Nick Downing [Mon, 24 Jan 2022 02:20:17 +0000 (13:20 +1100)]
Fix bug with not marking array or object dirty after deleting a property

2 years agoFix several bugs with opening database when the root straddles a block boundary
Nick Downing [Sun, 23 Jan 2022 06:08:37 +0000 (17:08 +1100)]
Fix several bugs with opening database when the root straddles a block boundary

2 years agoUse (Transaction|LazyValue).get_json() instead of logjson.logjson_to_json() and ...
Nick Downing [Sun, 23 Jan 2022 01:54:03 +0000 (12:54 +1100)]
Use (Transaction|LazyValue).get_json() instead of logjson.logjson_to_json() and (Transaction|LazyValue).set_json() instead of Transaction.json_to_logjson()

2 years agoFix bug with modifying the cache values causing Transaction.rollback() to fail
Nick Downing [Thu, 13 Jan 2022 04:14:53 +0000 (15:14 +1100)]
Fix bug with modifying the cache values causing Transaction.rollback() to fail

2 years agoFix bug in Database.Transaction() of not awaiting this.mutex.acquire()
Nick Downing [Thu, 13 Jan 2022 04:00:46 +0000 (15:00 +1100)]
Fix bug in Database.Transaction() of not awaiting this.mutex.acquire()

2 years agoImplement log file rotation
Nick Downing [Sat, 8 Jan 2022 03:30:23 +0000 (14:30 +1100)]
Implement log file rotation

2 years agoAdd immediate flush when Database.write_list_len exceeds Database.block_size, fix...
Nick Downing [Sat, 8 Jan 2022 01:39:42 +0000 (12:39 +1100)]
Add immediate flush when Database.write_list_len exceeds Database.block_size, fix bugs in kick() routine to respect cache pinning and improve synchronization

2 years agoImplement Database.block_size (previously hard coded to 0x1000), change Database...
Nick Downing [Sat, 8 Jan 2022 00:53:18 +0000 (19:53 -0500)]
Implement Database.block_size (previously hard coded to 0x1000), change Database.eof1 to Database.eof + Database.write_list_len

2 years agoMake default_value be in JSON not logjson format (faster and more convenient)
Nick Downing [Fri, 7 Jan 2022 06:55:23 +0000 (17:55 +1100)]
Make default_value be in JSON not logjson format (faster and more convenient)

2 years agoAdd json_to_logjson and logjson_to_json command line utilities
Nick Downing [Fri, 7 Jan 2022 06:41:05 +0000 (17:41 +1100)]
Add json_to_logjson and logjson_to_json command line utilities

2 years agoFix minor bugs
Nick Downing [Fri, 7 Jan 2022 06:20:33 +0000 (17:20 +1100)]
Fix minor bugs

2 years agoAdd packaging
Nick Downing [Fri, 7 Jan 2022 05:31:14 +0000 (16:31 +1100)]
Add packaging

2 years agoRename abstract parent class of Lazy(Array|Object) from Lazy to LazyValue
Nick Downing [Fri, 7 Jan 2022 05:29:18 +0000 (16:29 +1100)]
Rename abstract parent class of Lazy(Array|Object) from Lazy to LazyValue

2 years agoBreak into separate source files per object
Nick Downing [Fri, 7 Jan 2022 05:27:22 +0000 (16:27 +1100)]
Break into separate source files per object

2 years agoMove a.mjs, b.mjs, json and logjson files into /tests
Nick Downing [Fri, 7 Jan 2022 05:18:29 +0000 (16:18 +1100)]
Move a.mjs, b.mjs, json and logjson files into /tests

2 years agoMake writing use blocks >= 16 kbytes where possible
Nick Downing [Fri, 7 Jan 2022 05:10:43 +0000 (16:10 +1100)]
Make writing use blocks >= 16 kbytes where possible

2 years agoAdd write list
Nick Downing [Fri, 7 Jan 2022 05:02:08 +0000 (16:02 +1100)]
Add write list

2 years agoAdd read cache
Nick Downing [Fri, 7 Jan 2022 04:10:16 +0000 (15:10 +1100)]
Add read cache

2 years agoMake snapped references to Lazy(Array|Object) in the tree be stored as the Lazy(Array...
Nick Downing [Fri, 7 Jan 2022 00:07:54 +0000 (11:07 +1100)]
Make snapped references to Lazy(Array|Object) in the tree be stored as the Lazy(Array|Object) ref directly rather than [ptr, len, ref], since ptr, len not used

2 years agoRename LazyXXX.flush() to LazyXXX.commit(), make LazyXXX.commit() write itself to...
Nick Downing [Thu, 6 Jan 2022 23:52:14 +0000 (10:52 +1100)]
Rename LazyXXX.flush() to LazyXXX.commit(), make LazyXXX.commit() write itself to disk instead of relying on parent to do so (because there is no longer any need for special handling of the root object to put < > around it since the < > will now surround a [ptr, len] record rather than the root object itself), fix semantics of LazyArray.delete() and LazyArray.pop() so length behaves like Array

2 years agoRemove the awkward ptr === -1 logic in commit(), flush() routines, since it is now...
Nick Downing [Thu, 6 Jan 2022 23:27:07 +0000 (10:27 +1100)]
Remove the awkward ptr === -1 logic in commit(), flush() routines, since it is now guaranteed that if ptr === -1 the subtree must have been dirty and written

2 years agoWhen placing a Lazy(Array|Object) into the tree, copy its [ptr, len] into the parent...
Nick Downing [Thu, 6 Jan 2022 23:24:56 +0000 (10:24 +1100)]
When placing a Lazy(Array|Object) into the tree, copy its [ptr, len] into the parent if available, this means that if a subtree is moved from one spot to another the subtree will only need to be rewritten if modified as well (if subtree is not modified, parent can just use a reference to the existing on-disk subtree)

2 years agoStore the [ptr, len] reference in the Lazy(Array|Object) instead of dirty flag (when...
Nick Downing [Thu, 6 Jan 2022 23:17:16 +0000 (10:17 +1100)]
Store the [ptr, len] reference in the Lazy(Array|Object) instead of dirty flag (when set to null instead of [ptr, len] it means that the contents are dirty)

2 years agoRationalize how root is stored to make it more like an item of an array or object...
Nick Downing [Thu, 6 Jan 2022 22:50:51 +0000 (09:50 +1100)]
Rationalize how root is stored to make it more like an item of an array or object (if primitive, stored directly, otherwise as a [ptr, len] reference array)

2 years agoImplement Database.(read|write)() to reduce duplication and allow caching later
Nick Downing [Thu, 6 Jan 2022 08:32:13 +0000 (19:32 +1100)]
Implement Database.(read|write)() to reduce duplication and allow caching later

2 years agoMake a Transaction object, temporarily removes caching ability (Transaction object...
Nick Downing [Thu, 6 Jan 2022 01:44:17 +0000 (12:44 +1100)]
Make a Transaction object, temporarily removes caching ability (Transaction object and all cached read/write data is thrown away at the end of a transaction)

2 years agoMake LazyXXX an inner object of Database, so that LazyXXX.database is set at construc...
Nick Downing [Wed, 5 Jan 2022 03:21:08 +0000 (14:21 +1100)]
Make LazyXXX an inner object of Database, so that LazyXXX.database is set at construction time, doesn't change thereafter, and must match when building trees

2 years agoRename (F|f)ile to (D|d)atabase
Nick Downing [Wed, 5 Jan 2022 03:08:43 +0000 (14:08 +1100)]
Rename (F|f)ile to (D|d)atabase

2 years agoReinstate primitive object optimization
Nick Downing [Wed, 5 Jan 2022 01:43:40 +0000 (12:43 +1100)]
Reinstate primitive object optimization

2 years agoImplement dirty flag, to prepare for reinstating primitive object optimization
Nick Downing [Wed, 5 Jan 2022 01:41:47 +0000 (12:41 +1100)]
Implement dirty flag, to prepare for reinstating primitive object optimization

2 years agoAllow for nonexistent root item, make get() default value be written back
Nick Downing [Wed, 5 Jan 2022 01:21:36 +0000 (12:21 +1100)]
Allow for nonexistent root item, make get() default value be written back

2 years agoGet lazy writing working
Nick Downing [Wed, 5 Jan 2022 01:08:21 +0000 (12:08 +1100)]
Get lazy writing working

2 years agoTemporarily remove optimization of writing primitive JSON values directly, so that...
Nick Downing [Wed, 5 Jan 2022 00:24:32 +0000 (11:24 +1100)]
Temporarily remove optimization of writing primitive JSON values directly, so that all children of array or object consist of a [ptr, len] redirection record

2 years agoGet lazy reading working
Nick Downing [Tue, 4 Jan 2022 11:29:09 +0000 (22:29 +1100)]
Get lazy reading working

2 years agoPut methods into a class called logjson.File
Nick Downing [Tue, 4 Jan 2022 08:28:58 +0000 (19:28 +1100)]
Put methods into a class called logjson.File

2 years agoInitial commit, can read and write a JSON tree in log style and append to a log
Nick Downing [Tue, 4 Jan 2022 04:15:24 +0000 (15:15 +1100)]
Initial commit, can read and write a JSON tree in log style and append to a log