moveable_pool.git
7 years agoDistinguish between swap reservation and occupation in "new core ..." printouts
Nick Downing [Sat, 1 Jun 2019 10:44:05 +0000 (20:44 +1000)]
Distinguish between swap reservation and occupation in "new core ..." printouts

7 years agoTidy up the actual_old_paras/actual_old_blocks stuff and make the test script include...
Nick Downing [Sat, 1 Jun 2019 05:07:40 +0000 (15:07 +1000)]
Tidy up the actual_old_paras/actual_old_blocks stuff and make the test script include all the information used to calculate success values (for added checking)

7 years agoChange all byte sizes to paras, BLOCK_SIZE to BLOCK_PARAS and so on, make core_block_...
Nick Downing [Sat, 1 Jun 2019 02:05:26 +0000 (12:05 +1000)]
Change all byte sizes to paras, BLOCK_SIZE to BLOCK_PARAS and so on, make core_block_mem/swap_block_mem use one int per simulated para not one uint8_t per byte

7 years agoSimplify hashing function to eliminate wraparound carry and use of long long
Nick Downing [Sat, 1 Jun 2019 00:21:47 +0000 (10:21 +1000)]
Simplify hashing function to eliminate wraparound carry and use of long long

7 years agoFix some defines and conditional compiles that got lost in the ifdef-generation
Nick Downing [Fri, 31 May 2019 15:19:25 +0000 (01:19 +1000)]
Fix some defines and conditional compiles that got lost in the ifdef-generation

7 years agoTurn things around so that swap is written from top down
Nick Downing [Fri, 31 May 2019 15:05:56 +0000 (01:05 +1000)]
Turn things around so that swap is written from top down

7 years agoRemove first/last transfer optimization, it will be done differently
Nick Downing [Fri, 31 May 2019 13:28:28 +0000 (23:28 +1000)]
Remove first/last transfer optimization, it will be done differently

7 years agoMake victim_core_blocks be calculated on the fly, keep only victim_swap_blocks
Nick Downing [Fri, 31 May 2019 13:17:23 +0000 (23:17 +1000)]
Make victim_core_blocks be calculated on the fly, keep only victim_swap_blocks

7 years agoRe-enable all diagnostic printfs
Nick Downing [Fri, 31 May 2019 13:13:10 +0000 (23:13 +1000)]
Re-enable all diagnostic printfs

7 years agoRemove support for INODE_SWAP, make PREALLOCATE_SWAP always, make PREALLOCATE_CORE...
Nick Downing [Wed, 29 May 2019 13:32:00 +0000 (23:32 +1000)]
Remove support for INODE_SWAP, make PREALLOCATE_SWAP always, make PREALLOCATE_CORE when INDIRECT_CORE (have done this automatically, adds diff --ifdef scripts)

7 years agoAdd forgotten file
Nick Downing [Wed, 29 May 2019 13:05:36 +0000 (23:05 +1000)]
Add forgotten file

7 years agoImplement binary format script (for use within kernel, where there's no scanf)
Nick Downing [Sat, 18 May 2019 02:34:56 +0000 (12:34 +1000)]
Implement binary format script (for use within kernel, where there's no scanf)

7 years agoMore printf and unnecessary header removal, improve preprocessed blank lines
Nick Downing [Sat, 18 May 2019 01:23:12 +0000 (11:23 +1000)]
More printf and unnecessary header removal, improve preprocessed blank lines

7 years agoImplement FUZIX build, reduces block sizes, simplifies hash function, fixes rassert...
Nick Downing [Sat, 18 May 2019 00:17:59 +0000 (10:17 +1000)]
Implement FUZIX build, reduces block sizes, simplifies hash function, fixes rassert issues

7 years agoRemove all non-essential printfs (for code size reduction in FUZIX build)
Nick Downing [Sat, 18 May 2019 00:14:01 +0000 (10:14 +1000)]
Remove all non-essential printfs (for code size reduction in FUZIX build)

7 years agoSeparate out block allocation bitmap stuff into a reuseable module block_pool.c
Nick Downing [Thu, 9 May 2019 12:06:06 +0000 (22:06 +1000)]
Separate out block allocation bitmap stuff into a reuseable module block_pool.c

7 years agoRename pool_realloc() to pool_alloc() with a mode argument instead of ad-hoc schemes...
Nick Downing [Sun, 5 May 2019 11:55:47 +0000 (21:55 +1000)]
Rename pool_realloc() to pool_alloc() with a mode argument instead of ad-hoc schemes like sending an offset of INT_MIN or not filling in the move_up() routine

7 years agoNew combined routine that does all of pool_alloc(), pool_realloc(), pool_realloc_base...
Nick Downing [Sun, 5 May 2019 03:31:21 +0000 (13:31 +1000)]
New combined routine that does all of pool_alloc(), pool_realloc(), pool_realloc_base() and does the best it can with whatever move()/move_up() routines found

7 years agoCombine non-moveable with moveable pool, define both head->move and head->move_up...
Nick Downing [Mon, 22 Apr 2019 07:22:10 +0000 (17:22 +1000)]
Combine non-moveable with moveable pool, define both head->move and head->move_up to enable moveability, or just head->move for ordinary malloc-like behaviour

7 years agoBack out change from commit f4338d0 to process_avail, as process_avail was already...
Nick Downing [Mon, 22 Apr 2019 06:23:04 +0000 (16:23 +1000)]
Back out change from commit f4338d0 to process_avail, as process_avail was already defined to include the indirect blocks so no need to estimate their removal

7 years agoMove swap space tests into do_swap_out(), re-evaluate for each victim to be less...
Nick Downing [Mon, 22 Apr 2019 05:26:09 +0000 (15:26 +1000)]
Move swap space tests into do_swap_out(), re-evaluate for each victim to be less conservative in the inode swap case (allow only 2 blocks for partial indirect)

7 years agoDo proper computations about disk free space including possible direct and indirect...
Nick Downing [Sun, 21 Apr 2019 11:38:36 +0000 (21:38 +1000)]
Do proper computations about disk free space including possible direct and indirect blocks (should revisit to be less conservative in transfer sizes by estimating each victim separately so we don't need to allow for so many partial direct/indirect blocks upfront), get NDEBUG working again (doesn't fill core with 0xaa)

7 years agoMore strictly declare variables at start of function, for ack
Nick Downing [Sun, 21 Apr 2019 05:38:47 +0000 (15:38 +1000)]
More strictly declare variables at start of function, for ack

7 years agoFix inode leak caused by the f_trunc() modifications
Nick Downing [Thu, 11 Apr 2019 13:26:26 +0000 (23:26 +1000)]
Fix inode leak caused by the f_trunc() modifications

7 years agoOptimize out the classic or moveable allocator when not needed
Nick Downing [Thu, 11 Apr 2019 12:41:26 +0000 (22:41 +1000)]
Optimize out the classic or moveable allocator when not needed

7 years agoFix several more minor bugs, add gen.sh which pre-runs the C preprocessor to generate...
Nick Downing [Thu, 11 Apr 2019 12:00:05 +0000 (22:00 +1000)]
Fix several more minor bugs, add gen.sh which pre-runs the C preprocessor to generate 6 specialized versions of the code

7 years agoFix INDIRECT_CORE with INODE_SWAP combination
Nick Downing [Thu, 11 Apr 2019 11:12:55 +0000 (21:12 +1000)]
Fix INDIRECT_CORE with INODE_SWAP combination

7 years agoCommit a forgotten file
Nick Downing [Thu, 11 Apr 2019 10:25:57 +0000 (20:25 +1000)]
Commit a forgotten file

7 years agoFirst cut at inode swapping, works, may have an inode leak somewhere
Nick Downing [Tue, 9 Apr 2019 14:10:56 +0000 (00:10 +1000)]
First cut at inode swapping, works, may have an inode leak somewhere

7 years agoExpand out the swap_read()/swap_write() functions inline, since they were only called...
Nick Downing [Tue, 9 Apr 2019 10:13:52 +0000 (20:13 +1000)]
Expand out the swap_read()/swap_write() functions inline, since they were only called once, and they were supposed to be abstract but now the abstract part is done by core_to_swap_copy()/swap_to_core_copy() and the rest is well defined -- makes core.h/swap.h and core.c/swap.c identical again except for core<->swap

7 years agoChange order of adjusting process_core_blocks/process_swap_blocks/victim_core_blocks...
Nick Downing [Sun, 7 Apr 2019 06:12:47 +0000 (16:12 +1000)]
Change order of adjusting process_core_blocks/process_swap_blocks/victim_core_blocks/victim_swap_blocks to reduce computation

7 years agoMove PREALLOCATE_CORE/PREALLOCATE_SWAP defines into the respective header files and...
Nick Downing [Sun, 7 Apr 2019 05:27:35 +0000 (15:27 +1000)]
Move PREALLOCATE_CORE/PREALLOCATE_SWAP defines into the respective header files and define victim_core_size/victim_swap_size there, remove old victim_xxx_item

7 years agoMore optimal handling of incomplete last block in swap_read()/swap_write()
Nick Downing [Sat, 6 Apr 2019 03:31:15 +0000 (14:31 +1100)]
More optimal handling of incomplete last block in swap_read()/swap_write()

7 years agoFix a bug in undo successful process realloc, fix a bug in number of blocks to transf...
Nick Downing [Sat, 6 Apr 2019 03:17:07 +0000 (14:17 +1100)]
Fix a bug in undo successful process realloc, fix a bug in number of blocks to transfer for swap_read() and swap_write(), fix minor inconsistencies

7 years agoMove core_block_mem and swap_block_mem into pool_test_run.c and make core/swap memory...
Nick Downing [Fri, 5 Apr 2019 23:11:14 +0000 (10:11 +1100)]
Move core_block_mem and swap_block_mem into pool_test_run.c and make core/swap memory accessor functions abstract, make core/swap move do incomplete last block

7 years agoGet rid of the DEDICATED_xxx() macros in favour of more optimal inline code
Nick Downing [Sat, 6 Apr 2019 02:47:04 +0000 (13:47 +1100)]
Get rid of the DEDICATED_xxx() macros in favour of more optimal inline code

7 years agoGet rid of the dedicated and per-process core/swap item, just use per-process
Nick Downing [Fri, 5 Apr 2019 11:34:32 +0000 (22:34 +1100)]
Get rid of the dedicated and per-process core/swap item, just use per-process

7 years agoImplement partial transfer to/from swap for incomplete last block pool_swap
Nick Downing [Thu, 4 Apr 2019 11:20:43 +0000 (22:20 +1100)]
Implement partial transfer to/from swap for incomplete last block

7 years agoMake swap reading/writing take bytes not blocks, allowing incomplete last block
Nick Downing [Thu, 4 Apr 2019 10:20:03 +0000 (21:20 +1100)]
Make swap reading/writing take bytes not blocks, allowing incomplete last block

7 years agoMake only the active (bytewise not blockwise) portion of process be hashed
Nick Downing [Thu, 4 Apr 2019 10:01:31 +0000 (21:01 +1100)]
Make only the active (bytewise not blockwise) portion of process be hashed

7 years agoMake process hash init and verify be bytewise rather than blockwise
Nick Downing [Thu, 4 Apr 2019 09:49:17 +0000 (20:49 +1100)]
Make process hash init and verify be bytewise rather than blockwise

7 years agoSimplify the calling convention of the hash init and verify routines
Nick Downing [Thu, 4 Apr 2019 09:21:18 +0000 (20:21 +1100)]
Simplify the calling convention of the hash init and verify routines

7 years agoMove the hash generation and verification stuff into process_test_run.c
Nick Downing [Thu, 4 Apr 2019 09:12:30 +0000 (20:12 +1100)]
Move the hash generation and verification stuff into process_test_run.c

7 years agoChange process.blocks to process.size, do rounding and shifting at each access
Nick Downing [Thu, 4 Apr 2019 09:07:11 +0000 (20:07 +1100)]
Change process.blocks to process.size, do rounding and shifting at each access

7 years agoChange test script to refer to bytes, although everything still done in blocks
Nick Downing [Thu, 4 Apr 2019 08:05:40 +0000 (19:05 +1100)]
Change test script to refer to bytes, although everything still done in blocks

7 years agoIn process related stuff, change size to blocks and SIZE to BLOCKS
Nick Downing [Thu, 4 Apr 2019 07:46:28 +0000 (18:46 +1100)]
In process related stuff, change size to blocks and SIZE to BLOCKS

7 years agoImplement further macros DEDICATED_CORE_BASE() and DEDICATED_SWAP_LIMIT() which know...
Nick Downing [Sun, 31 Mar 2019 10:17:28 +0000 (21:17 +1100)]
Implement further macros DEDICATED_CORE_BASE() and DEDICATED_SWAP_LIMIT() which know whether the underlying items get updated during swap or whether they are preallocated at process creation time, and take a correction factor to apply in the latter case (by macro, so the correction is not evaluated in the former case)

7 years agoImplement higher level macros to access process->core_item vs process->pool_item...
Nick Downing [Sun, 31 Mar 2019 07:43:24 +0000 (18:43 +1100)]
Implement higher level macros to access process->core_item vs process->pool_item, victim->core_item vs victim_core_item and so on (depends on configuration)

7 years agoRemove some bogus assertions that don't work in preallocated modes, tidy up
Nick Downing [Sun, 31 Mar 2019 07:45:24 +0000 (18:45 +1100)]
Remove some bogus assertions that don't work in preallocated modes, tidy up

7 years agoChange direction of swap in/out to forward, so that core is read/written in deque...
Nick Downing [Sun, 31 Mar 2019 02:40:20 +0000 (13:40 +1100)]
Change direction of swap in/out to forward, so that core is read/written in deque fashion (when swapping out, pop off the base, when swapping in, push onto the base) whereas swap is read/written in file fashion (when swapping out, append to the limit, when swapping in, truncate the limit), allows a swap-to-inode mode

7 years agoAdd pool_realloc_base() and pool_realloc_base_moveable() to allow a region to be...
Nick Downing [Sat, 30 Mar 2019 12:39:13 +0000 (23:39 +1100)]
Add pool_realloc_base() and pool_realloc_base_moveable() to allow a region to be used like a deque, new space can be pushed onto and contents popped off start

7 years agoAdd inode test script
Nick Downing [Thu, 28 Mar 2019 12:30:27 +0000 (23:30 +1100)]
Add inode test script

7 years agoAdd UCP from FUZIX, with modifications to isolate FS stuff into new fuzix_fs.c
Nick Downing [Sun, 24 Mar 2019 02:38:30 +0000 (13:38 +1100)]
Add UCP from FUZIX, with modifications to isolate FS stuff into new fuzix_fs.c

7 years agoImplement simulated BLOCK_SIZE, hash create and check simulated block contents
Nick Downing [Sat, 23 Mar 2019 22:55:43 +0000 (09:55 +1100)]
Implement simulated BLOCK_SIZE, hash create and check simulated block contents

7 years agoMake n_blocks the primary size and table_size only given if indirect (was the other...
Nick Downing [Sat, 23 Mar 2019 21:46:23 +0000 (08:46 +1100)]
Make n_blocks the primary size and table_size only given if indirect (was the other way around, making block stuff use either table_size or n_blocks), fix bugs

7 years agoSubroutinize the hashing initialization and verification, to avoid duplication
Nick Downing [Sat, 23 Mar 2019 06:22:24 +0000 (17:22 +1100)]
Subroutinize the hashing initialization and verification, to avoid duplication

7 years agoPrint message like 'alloc NN NN true|false' and then perform the operation and then...
Nick Downing [Sat, 23 Mar 2019 05:50:59 +0000 (16:50 +1100)]
Print message like 'alloc NN NN true|false' and then perform the operation and then '... ok', so that we can see clearly what messages come from what operation

7 years agoRemove the reservation of spare space from pool object, core and swap modules
Nick Downing [Sat, 23 Mar 2019 04:44:49 +0000 (15:44 +1100)]
Remove the reservation of spare space from pool object, core and swap modules

7 years agoImplement INDIRECT_CORE, fix calculations so that PREALLOCATE_CORE make sense
Nick Downing [Tue, 19 Mar 2019 13:13:23 +0000 (00:13 +1100)]
Implement INDIRECT_CORE, fix calculations so that PREALLOCATE_CORE make sense

7 years agoFor direct swap, store data in swap_block_mem not swap_table_mem, also for core
Nick Downing [Tue, 19 Mar 2019 10:36:45 +0000 (21:36 +1100)]
For direct swap, store data in swap_block_mem not swap_table_mem, also for core

7 years agoAdd INDIRECT_SWAP option
Nick Downing [Tue, 19 Mar 2019 10:11:51 +0000 (21:11 +1100)]
Add INDIRECT_SWAP option

7 years agoInsert table_ in most core.c / swap.c functions e.g. core_alloc() becomes core_table_...
Nick Downing [Mon, 18 Mar 2019 10:24:39 +0000 (21:24 +1100)]
Insert table_ in most core.c / swap.c functions e.g. core_alloc() becomes core_table_alloc(), because there will soon be an allocator for random block storage

7 years agoImplement MOVEABLE_CORE and MOVEABLE_SWAP compile time options
Nick Downing [Sun, 17 Mar 2019 03:23:55 +0000 (14:23 +1100)]
Implement MOVEABLE_CORE and MOVEABLE_SWAP compile time options

7 years agoFix a bug of requiring a certain amount of free swap, when swap is preallocated
Nick Downing [Sun, 17 Mar 2019 03:16:54 +0000 (14:16 +1100)]
Fix a bug of requiring a certain amount of free swap, when swap is preallocated

7 years agoFix a bug of victim_core_size being used uninitialized without PREALLOCATE_CORE
Nick Downing [Sun, 17 Mar 2019 03:00:27 +0000 (14:00 +1100)]
Fix a bug of victim_core_size being used uninitialized without PREALLOCATE_CORE

7 years agoImplement PREALLOCATE_CORE and PREALLOCATE_SWAP compile time options
Nick Downing [Sun, 17 Mar 2019 02:24:46 +0000 (13:24 +1100)]
Implement PREALLOCATE_CORE and PREALLOCATE_SWAP compile time options

7 years agoImprove use of alloc()/realloc()/free() to remove allocs and reallocs of size 0
Nick Downing [Sat, 16 Mar 2019 22:27:26 +0000 (09:27 +1100)]
Improve use of alloc()/realloc()/free() to remove allocs and reallocs of size 0

7 years agoTidy up debugging code
Nick Downing [Sat, 16 Mar 2019 14:03:15 +0000 (01:03 +1100)]
Tidy up debugging code

7 years agoReplace core_item and swap_item in the process structure with just pool_item
Nick Downing [Sat, 16 Mar 2019 13:58:21 +0000 (00:58 +1100)]
Replace core_item and swap_item in the process structure with just pool_item

7 years agoKeep process and victim pool items in dedicated storage during swapping (should gener...
Nick Downing [Sat, 16 Mar 2019 13:47:56 +0000 (00:47 +1100)]
Keep process and victim pool items in dedicated storage during swapping (should generate smaller code, also ensures a process has only one pool item at a time)

7 years agoDon't track victim or in-swap processes in LRU list (should've thought of this)
Nick Downing [Sat, 16 Mar 2019 12:43:33 +0000 (23:43 +1100)]
Don't track victim or in-swap processes in LRU list (should've thought of this)

7 years agoUse process->size == -1 to mark free process not process->lru_item.prev == NULL
Nick Downing [Sat, 16 Mar 2019 11:36:31 +0000 (22:36 +1100)]
Use process->size == -1 to mark free process not process->lru_item.prev == NULL

7 years agoReverse swap addressing, so that swap is allocated from top down, and copies between...
Nick Downing [Fri, 15 Mar 2019 23:55:25 +0000 (10:55 +1100)]
Reverse swap addressing, so that swap is allocated from top down, and copies between memory and swap always go forward with respect to the actual backing store

7 years agoCombine in-core and in-swap LRU list, using global victim pointer to first swap
Nick Downing [Fri, 15 Mar 2019 13:21:48 +0000 (00:21 +1100)]
Combine in-core and in-swap LRU list, using global victim pointer to first swap

7 years agoMake the current victim be stored at head of in-swap LRU list instead of tail of...
Nick Downing [Fri, 15 Mar 2019 13:02:52 +0000 (00:02 +1100)]
Make the current victim be stored at head of in-swap LRU list instead of tail of in-core LRU list, strengthen invariants about core and/or swap membership

7 years agoUse #define to improve casting for lru_item vs process pointers
Nick Downing [Fri, 15 Mar 2019 07:59:39 +0000 (18:59 +1100)]
Use #define to improve casting for lru_item vs process pointers

7 years agoChange process_head/process_item to lru_head/lru_item which is more descriptive
Nick Downing [Fri, 15 Mar 2019 07:17:04 +0000 (18:17 +1100)]
Change process_head/process_item to lru_head/lru_item which is more descriptive

7 years agoProcess test, it is based on the pool test but involves two pools, one representing...
Nick Downing [Thu, 14 Mar 2019 14:31:39 +0000 (01:31 +1100)]
Process test, it is based on the pool test but involves two pools, one representing core and the other representing swap, and a new command 'run' which moves the given process fully into core, using an LRU algorithm to boot out some others

7 years agoInitial commit, seemingly correct malloc/realloc/free pool implementation, where...
Nick Downing [Mon, 11 Mar 2019 01:58:08 +0000 (12:58 +1100)]
Initial commit, seemingly correct malloc/realloc/free pool implementation, where the user supplies the control blocks (instead of them appearing at the start of an allocation), and existing allocations can be moved to make room as needed