First step towards single tokenization.
authorJakub Pawlowicz <contact@jakubpawlowicz.com>
Mon, 23 Mar 2015 20:36:07 +0000 (20:36 +0000)
committerJakub Pawlowicz <contact@jakubpawlowicz.com>
Sun, 12 Apr 2015 11:15:29 +0000 (12:15 +0100)
commitd72356a510980f274f18b1fddb72bd089d975cfe
tree1fe64623b35d8d84767e1229fc69996736da9b2a
parent6957bc71feaf76819dfa3ae53320992d87bd6a5b
First step towards single tokenization.

* Removes tokenization from all steps in favour to single at the beginning.
* Adds tokenization of properties value by value instead of all in one piece, e.g.
  `margin:0px 1px` gets tokenized into `margin`, `0px, and `1px`.
* Full tokenization means more detailed source maps too.
* Reworks override compactor and associated classes to use single tokenization.
* Simplifies override compactor code so it's much easier to understand.
* Adds loads of tests to override compactor and family.

Caveats / in progress:
* Shorthand compactor is turned off.
* Inherit compacting is gone (but will be back).
* Some multivalue background tests are failing.
* There's no way to turn off compacting (possible performance regression).
49 files changed:
lib/properties/break-up.js [new file with mode: 0644]
lib/properties/can-override.js [new file with mode: 0644]
lib/properties/compactable.js [new file with mode: 0644]
lib/properties/optimizer.js
lib/properties/override-compactor.js
lib/properties/populate-components.js [new file with mode: 0644]
lib/properties/processable.js [deleted file]
lib/properties/remove-unused.js [new file with mode: 0644]
lib/properties/restore-shorthands.js [new file with mode: 0644]
lib/properties/restore.js [new file with mode: 0644]
lib/properties/shallow-clone.js [new file with mode: 0644]
lib/properties/shorthand-compactor.js
lib/properties/token.js [deleted file]
lib/properties/validator.js
lib/properties/wrap-for-optimizing.js [new file with mode: 0644]
lib/selectors/extractor.js [moved from lib/properties/extractor.js with 71% similarity]
lib/selectors/metadata.js [new file with mode: 0644]
lib/selectors/optimization-metadata.js [new file with mode: 0644]
lib/selectors/optimizer.js
lib/selectors/optimizers/advanced.js
lib/selectors/optimizers/simple.js
lib/selectors/reorderable.js [moved from lib/properties/reorderable.js with 88% similarity]
lib/selectors/source-map-stringifier.js
lib/selectors/stringifier.js
lib/utils/extractors.js
lib/utils/splitter.js
lib/utils/stringify-tokens.js
test/binary-test.js
test/integration-test.js
test/properties/break-up-test.js [new file with mode: 0644]
test/properties/longhand-overriding-test.js [new file with mode: 0644]
test/properties/optimizer-test.js [new file with mode: 0644]
test/properties/override-compacting-test.js [new file with mode: 0644]
test/properties/populate-components-test.js [new file with mode: 0644]
test/properties/remove-unused-test.js [new file with mode: 0644]
test/properties/reorderable-test.js [deleted file]
test/properties/restore-shorthands-test.js [new file with mode: 0644]
test/properties/restore-test.js [new file with mode: 0644]
test/properties/shorthand-compacting-test.js [new file with mode: 0644]
test/properties/wrap-for-optimizing-test.js [new file with mode: 0644]
test/selectors/extractor-test.js [moved from test/properties/extractor-test.js with 56% similarity]
test/selectors/optimization-metadata-test.js [new file with mode: 0644]
test/selectors/optimizer-test.js
test/selectors/optimizers/simple-test.js
test/selectors/reorderable-test.js [new file with mode: 0644]
test/selectors/tokenizer-source-maps-test.js
test/selectors/tokenizer-test.js
test/source-map-test.js
test/utils/splitter-test.js