[update] Improved the tokenizing algorithm.

This commit is contained in:
2019-10-04 00:00:00 +00:00
parent e7ebe7ddef
commit 53295526b3
3 changed files with 41 additions and 51 deletions