Comment by nice_byte

Yeah I misused the term "token". What I really meant is that they should just be token separator and nothing more (i.e. once we encounter whitespace it means the current token has ended)