Comment by jrdres

Chinese apparently was originally always written vertically top-to-bottom. (And then columns would be right-to-left.) Modern Chinese just rotates everything except the characters themselves 90 degrees to the Latin order.

I also read that a few Chinese texts only make sense in vertical order: one had a pun where the characters read one way as separated characters, but as stacked was also a single character pun for something like a "crumbly cookie".