Moves cleaning up tokens out of tokenizer.
authorJakub Pawlowicz <contact@jakubpawlowicz.com>
Mon, 27 Oct 2014 20:46:17 +0000 (20:46 +0000)
committerJakub Pawlowicz <contact@jakubpawlowicz.com>
Mon, 8 Dec 2014 09:39:14 +0000 (09:39 +0000)
commitb06879dcbc3d32d19fb2bedc23af90b1473d109a
tree22e51992c5923b51cf0b7f6e2f04938f0f59b4c4
parente2ddaaa4aee2a393aafa869aaa6c28deac7fcb15
Moves cleaning up tokens out of tokenizer.

* Pulls out extracting logic into an utility class.
lib/selectors/optimizer.js
lib/selectors/optimizers/clean-up.js
lib/selectors/optimizers/simple.js
lib/selectors/tokenizer.js
lib/utils/extractors.js [new file with mode: 0644]
test/selectors/tokenizer-test.js