Comment by themafia
16 days ago
> Pascal style strings were much safer.
The limitations were brutal. Initially you could only have 255 bytes in a string. The length of a string and the size of the allocation are now separate and you may need to think about that unused memory in your design. The problem now doubles with the introduction of UTF-8. Your string size is in bytes and you need to track characters separately.
If you want to create an array of strings you either need to specify the length of all strings and accept the memory overhead or have an array of pointers to strings. If you use an array of pointers you may end up choosing to use the 'nil' value as a sentinel that means "end of list." So we're right back where we started.
--
Because someone decided to downvote this HN has limited the speed at which I can reply. This site is tragic and I'm fully done with it now. You can spread propaganda and poorly sourced zeitgeist and be among friends but if you try to have a genuine conversation about programming languages you are made to be unwelcome immediately. Screw this.
--
> No other data structure works like this.
The linked list.
> You can't mess this up in an array
C happily decomposes arrays into pointers. You can erase your length information from the type. This was an intentional decision.
> Strings are the only data structure that assume there will be a NULL at end.
Which is why almost every string API has a version that allows you to specify the maximum length. The fact that you can use a NUL doesn't mean you have to. Which is why the concept of "sentinel values" is broadly used in many types of applications you haven't considered here.
> You can spread propaganda and poorly sourced zeitgeist and be among friends but if you try to have a genuine conversation about programming languages you are made to be unwelcome immediately.
Indeed. And the ignorance of computing history in this discussion is particularly disturbing.
The context of this particular thread is "zero terminated string is ... computing's biggest mistake". This completely ignores the situation on the ground when C was developed. At the time, people were striving for a system programming language that sat above the level of assembly but was compact enough to run within the limited resources of the then emerging mini-computer systems. The PDP-11 on which C was developed was certainly not the first mini-computer, but it was among the earliest to have a regular enough instruction set and addressing model to make a general purpose, high-level system's language possible. These systems were extremely limited in memory; the PDP-11's instruction set is limited to directly addressing at most 64KiB (code and data) and many systems of the era were hardware limited to less than that. (Indeed, I regularly run an early version of Unix, including an early C compiler, on my PDP-11/05 which is maxed out at 56KiB [of actual core]). There was no way that even a brilliant engineer like Dennis Richie was going to be able to shoe-horn in "optional" types, or the mechanics of length-value strings into a compiler that has to run in such limited space, and produce code (e.g. the Unix kernel) that has to run in even less. The fact that strings and arrays are thin abstractions on top of pointers is both a brilliant compromise in design as well as a nod to then-prevalent assembly practice. It was the exactly kind of pragmatic decision that was needed to move computing along at the time. Of course the designs from this era are antiquated now. But they were not mistakes.
> This completely ignores the situation on the ground when C was developed.
A great many of those replying are many years short of having experienced anything like "the situation on the ground when C was developed". They simply have never known a day without hundreds of gigabytes or more of disk storage and 8G or more of RAM available for user processes after the OS consumes what it needs for its own work. They are "ignoring" because they simply have no basis for understanding.
The C code for strcpy is:
On a PDP-11 that is:
All those limitations were sorted out in 1978 with Modula-2 and open arrays, aka spans.
What about the UNIX and C folks propaganda of C being the first systems language, or always focusing on the original Pascal used for teaching and not everything else that followed up with Mesa, Modula-2, Ada, Object Pascal and friends, none of them with said limitations.
C was specifically developed to allow Unix to be ported.
It was a systems programming language and the first well known/successful one.
There was BCPL and then B before that, which is why the language is called "C".
Pascal was considered a teaching language, along with "Algorithms + Data Structures = Programs" by Wirth etc.
The UCSD P-system was one of the first "IDEs" and used Pascal and a bytecode interpreter of the compiled code.
Modula-2 was barely available in the early 1980s.
Ada was mired in MIL-SPEC and expensive compilers etc.
People used FORTRAN for scientific programming, C for most everything else in the non-IBM mainframe world.
Successful systems languages trace back all the way to JOVIAL in 1958.
You missed quite a few between JOVIAL, and C being adopted outside Bell Labs.
Modula-2 was as widely available as C was outside UNIX and universities with access to UNIX source code.
It took a while for proper C to actually be "used for everything else", until the early 1990s actually, and by then anyone sensible would be much better with Typescript for C, aka C++.
9 replies →
> You can erase your length information from the type. This was an intentional decision.
Well yes, but given the number of security issues the argument is that it was in retrospect the wrong decision.
> Your string size is in bytes and you need to track characters separately
No worse than C strings then.
>The problem now doubles with the introduction of UTF-8. Your string size is in bytes and you need to track characters separately.
That isn't really a problem.
The problem with null-terminated strings is specifically what happens when you reach the end of the allocated array and there ISN'T a NULL character.
Every string function is designed to keep going until it finds the NULL character, so if a hacker gets rid of the NULL character, he can exploit pretty much any standard string manipulation function being used elsewhere in the program to manipulate whatever memory comes AFTER the string data structure.
No other data structure works like this. You can't mess this up in an array, because no function that manipulates arrays is just going to keep going until there is a null. That would be stupid because it would require users of the function to add a NULL to the end of their arrays before passing it to the function, so instead we just pass the size of the array to everything. Strings are the only data structure that assume there will be a NULL at end.
By the way, I read once that if you use UTF-32 every code point will be 4 bytes, constantly, but even then a single code point isn't necessarily a single character. Text is just complicated.
> No other data structure works like this.
In C most data structures work like this, you keep going until you find NUL (character) or NULL (pointer). E.g. Strings, array of pointers, linked lists, etc. Of course you can add length to most of those, but it isn't the canonical/traditional way of doing things.
That can't be true. If you have an array of pointers it can be terminated in NULL. But an array of integers can't have a NULL value, since NULL would probably be just 0 which is a normal integer.
The null in a linked list is the null in the .next field, right? That's the way you would implemented linked lists independent of language. It's not the .value that is null.
A string is an array of characters (well, for characters representable in one byte at least) that has a specific value to represent the end of string.
It would be like if Int::MAX was reduced by 1 to make space for an Int:NUL constant that represented the end of an integer array. Or if you were creating your own ENUM, let's say for NORTH, SOUTH, EAST, WEST, and you added a fifth enumeration called Direction.NUL for use in arrays.
1 reply →
> No other data structure works like this. You can't mess this up in an array, because no function that manipulates arrays is just going to keep going until there is a null.
This is patently false. Sentinel markers are used widely in array types. Consider GNU's getopt_long() function, a mainstay in GNU tools:
The argument longopts must be an array of [struct option] structures, one for each long option. Terminate the array with an element containing all zeros.
> Every string function is designed to keep going until it finds the NULL character, so if a hacker gets rid of the NULL character,
What sort of situation are you envisioning where a hacker can remove the sentinel (in the case of nul-termination) but not modify the length bytes (in the case of fat pointers)?
A situation in which a string is manipulated with buggy code that can remove the sentinel, e.g. the program uses strncpy, there is a bug in how it uses it, the hacker exploits the bug.
By contrast it's pretty unlikely for buggy code to mess the length. Add an element? +1. Remove an element? -1. Number of elements larger than capacity? Allocate a new array. Not much room for error.
If I zero out the destination buffer of a strcpy, and the string is longer than the destination buffer I will run into a buffer overflow problem despite every byte being a zero byte. The absence or presence of the zero byte doesn't seem to be the deciding factor.