Comment by lewisl9029

Perhaps another reason for this is that Javascript is inherently unsafe. By relying on these small rigorously tested libraries they are avoiding the need to test their own code, and thus avoiding basic null check or conversion errors.

Other languages handle this with compilers, may be strongly typed, and/or have features that don't allow nulls. Javascript doesn't exactly have that luxury. So maybe it makes sense in Javascript, where it wouldn't in other languages (though one could argue this is a flaw in the language design).

edit: to be clear I'm not really defending the practice, but trying to give a little perspective.

> Yes they are, in the same way that a book in which every page consists of a single word is easier to understand than one with more content per page.

A more apt metaphor would be separating a book into many small paragraphs that each serves a single purpose makes the book easier to understand. Regardless, the book metaphor misses a lot of the nuance.

Of course taking the approach to an extreme would be detrimental. However, a vast majority of these small modules that are actually widely used consist of functions that may seem trivial at first sight, but actually contain a lot of corner cases and special considerations that a naive inline implementation could miss.

Sure, there are also plenty of trivial one-line modules on npm that don't fit into a such a description, but those are simply side effects of the unprecedented popularity of the platform and its completely open nature, and shouldn't be used to infer any kind of general trend towards "hypermodularisation" because very few reasonable developers would ever import them into their projects.

> By focusing on the small-scale complexity to such an extreme, you've managed to make the whole system much harder to understand

Well put. I've noticed a curious blind spot in how people account for complexity: we count the code that 'does' things much more than the code that glues those things together. This distorts our thinking about system complexity. A cost/benefit analysis that doesn't consider all the costs isn't worth much.

An example is when people factor complex code into many small functions, then say it's much simpler because smaller functions are easier to understand. In fact this may or may not be true, and often isn't. To get a good answer you must consider the complexity you've added—in this case, that of the new function declarations and that of the calls to them—not just the complexity you've removed. But it's hard to consider what you don't see. Why is it so easy not to see things like this? I think it's our ideas about programming, especially our unquestioned assumptions.

The complexity of glue code goes from being overlooked to positively invisible when it gets moved into things like configuration files. Those are no longer seen as part of the system at all. But of course they should be.

And to add to the mess, until npm introduced deduplication every dependency recursively included it's own dependencies as sub folders, so you get a bajillion tiny dependencies over and over (and sometimes with different versions to boot).

Frankly some of the comments I'm seeing really do reinforce his point: people really don't know how to program. I suspect this is it the flipside of Javascript being popular and accessible - people can churn out products without really knowing what they are doing...

Sorry for the nitpick, but "hyperabstraction"?

left-pad isn't even slightly abstract.

JS does not have an Integer type, so (x > 0) does not work. Now if you add all those checks necessary to see whether you're actually dealing with an integer here, you get to a point where you cannot be sure anymore whether there aren't any weird type coercion issues there and need to start writing tests. Suddenly your whole positive integer check took 1h to write and you still cannot be sure whether there's something you didn't think of. I'd rather use an external library for that that's properly unit tested.

Not defending javascript because I dislike it immensely, but with javascript being a dynamically typed language, and with the way it handles implicit conversions between types, and with trickiness around NaN, Infinity and null objects, there are sufficient edge cases that writing a correct function to test if some variable contains a positive integer is not as trivial as it is with more sane languages, and is likely to trip up even experienced developers if they miss one or more of those edge cases.

Having pre-written modules that account for all edge cases that have been battle-tested by hundreds, thousands, or even millions of users does have merit.

The main problem here is with the Javascript language, which makes code evolve to things like 'is-positive-integer'.

If you want to build is-bigger-than-5 yourself, I recommend pulling in fivejs[1] as a dependency.

[1] https://github.com/jackdcrawford/five

add-1-to-a-number

This seems to be a very common example in educational materials about how to create a function in some programming language. Perhaps they should all be prefaced with "NOTE: you should not actually make such a function, because its triviality means that every time you use it you will be increasing the complexity of your code", or a longer discussion on when abstraction in general is appropriate and when it is not.

Ditto for any other abstraction --- I've found there's too much of an "abstraction is good so use it whenever you can" theme in a lot of programming/software design books and not enough "why abstraction can be bad" sort of discussion, which then leads to the "you can never have too much" mentality and the resultant explosion of monstrous complexity. Apparently, trusting the reader to think about the negatives and exercise discretion didn't work so well...

It doesn't.

Of course we have a number of packages that are truly trivial and should probably be inlined. Npm is a completely open platform. Anyone can upload any module they feel like. That doesn't mean anyone else will feel the need to use them.

I think you'll find that the vast majority of small modules that are widely used are ones that also cover obscure corner cases and do benefit from widespread use and testing.

No, you see "is-bigger-than-5" clearly does 4 different things. You need these modules:

module.exports = exec("===");

module.exports = bigger(a, b){return a > b};

module.exports = () => return "than";

module.exports = (a) => a.search('ABCD...');

.. or something like that. These are all plugins of course.

Do you seriously think that "word starts with a capital letter" is an easy function to write?

I feel like you haven't spent enough time with Unicode.

The three cases you state aren't nearly as hairy as determining whether something is a positive integer.

Someone might do something like ~~n to truncate a float. That works fine until you go above 32 bits. Math.trunc should be used instead. Someone might do something like n >>> 0 === n, and using any bitwise operators will always bake in the assumption that the number is 32 bits. Do you treat negative 0 as a negative integer? Detecting negative 0 is hairy. So, to avoid bad patterns, it makes sense.

For is-bigger-than-5(x), x > 5 is not hairy. For add-1-to-a-number(n), n + 1 is not hairy.

For word-starts-with-capital-letter(word)? That one is actually pretty hairy. There are programmers that would write a regular expression /^[A-Z]/, or check the charCode of the first character being within the A-Z range, amongst other solutions. An appropriate solution, however, would be (typeof word == "string" && word.length && word[0] == word[0].toUpperCase() && word[0].toUpperCase() != word[0].toLowerCase()), because the toUpperCase and toLowerCase methods are unicode-aware, and presently you can't access unicode properties using Javascript's flavor of regular expressions.

People have started to point out the existence of modules in this very vein. And, of course, write them. With two such together, you can probably get is-maybe-5 .

* https://news.ycombinator.com/item?id=11359302

If those functions are reusable or complex enough (partly due to deficiencies in javascript the language, sure) then why not?

>add-1-to-a-number

i++

The isArray package has 72 dependent NPM packages, it's certainly not undiscoverable. The leftpad package gets 5000 downloads a month, that's quite a bit of testing for edge cases, compared to the none that I would have gotten had I implemented this myself.

Intuitively, this thread wouldn't exist if your assertion were correct.

Discoverability though is... poor seems like an argument for improving discoverability, not against having small modules.

Can we ever achieve this?

We can't exactly search the code to find out what it does, otherwise you'd basically be reading all the code to discover it's true behavior, which negates the usefulness of modules in the first place...

Any search will have to rely on developer-made documentation, and/or meta data. This is great in theory, but documentation is rarely well maintained, and/or someone changes the code but neglects to update the documentation.

This leaves us with the situation we have today. A search that somewhat works kindof, and mostly you rely on googling the feature you need and looking for what seems to be the most popular choice.

I'm not sure how this situation can be made better, especially if we continue down a path of having all these "one-liner" libraries/modules that anyone can make and stick out there into the unsuspecting world. When I need a square root function, and my search returns 800 one-liner modules, how am I supposed to pick one that I know is reasonably well done, and does what it says it will do, without looking under the hood - you'll end up just picking whatever seems to be popular...

You're totally right. Most developers don't develop this way, and that makes the benefits of this approach far less pronounced than they would be if they did. It also doesn't negate the benefits entirely, of course.

Another benefit of small well understood components is that they are easier to write tests for. Do these npm modules have good test coverage? Did the leftpad module have a unit test that would have caught the issue?

I think the gist of this whole discussion ( at least the OMG WHY?!?! ) part, can be easily explained by an excerpt from your example comment that sums up in a nutshell the all too pervasive mindset I've seen over the years:

"...LOC is pretty much irrelevant. It doesn't matter if the module is one line or hundreds. It's all about containing complexity. Think of node modules as lego blocks. You don't necessarily care about the details of how it's made. All you need to know is how to use the lego blocks to build your lego castle. By making small focused modules you can easily build large complex systems without having to know every single detail of how everything works..."

By LOC he's referencing 'ye ole Lines of Code paradigm, and trying to make the point that in the end it just doesn't measure up to the prime directive of "containing complexity".

... and that's where I beg to differ.

What I think is being completely overlooked here is the net cost of abstracting away all that complexity... It's performance.

Every extra line of code, extraneous function call, unnecessary abstraction ( I'm looking at you promise ), every lazy inclusion of an entire library for the use of a shortcut to document.getElementById -- these all add unnecessary costs from the network down to the cpu.

And who pays these costs?

The end users of your slow, memory hogging, less-than-ideal set of instructions to the cpu that you call an application... but hey, it's easier for you, so there's that.

> Maybe some people have so little to do that they can author or find a relevant package for every single ~10 line function they need to use in their code and then spend countless commits bumping project-versions and updating package.json files. I have no idea how they get work done though...

Rather than copy paste someone else's unmaintained thing and handle all the bugs, or write their own code and unit tests, they use the same library as a few thousand other people?

How is it superior? Simple.

How do I systematically make sure that I have the latest version of every stackoverflow code snippet? If it's a new post, it may not have all the edge cases fixed yet. So now I have to check back on each of the X number of snippets I've copied.

In the npm approach, I can easily tell if there's a new version. For prod, I can lock to a specific version, but in my test environment, I can use ^ to get newer versions and test those before I put them in production.

If the edge case of new version of a package breaks my code, I've learned that I'm missing a unit test. Plus, the question isn't whether this bad thing might happen on occasion, the question is whether this approach is, on balance, superior to cutting and pasting random code snippets into my code. I think the downside of the npm approach is less than the downside of the copypasting from stackoverflow approach.

And every moderately useful npm package I've looked at has very good to great documenation.

With the advent of ES6 modules's selective imports and tree-shaking, that's definitely quickly becoming a better approach. With the old CommonJS modules, you need to be concerned about overall code size, which is where small, single purpose modules excel, and why this approach has proliferated to this degree.

People need modules like leftpad to tick the "maintains moderately popular open source project" checkbox. Instant hirability.

I don't want to claim that it would be a directly calculated career move, more like starting a blog: you admire a good blog, you want to be more like the blogger, you start your own one. On the dim chance that it will become both good and not abandoned after the third post. Nanomodules can be just like that, the air guitar of would-be rockstar coders. This is certainly not the only reason for their existence (and even the air-guitar aspect has a positive: low barrier of entry that might lead to more serious contribution), but the discussion would be incomplete if it was ignored.

if leftpad were in the node stdlib, would you still be making the same point about knowing computer science and using leftpad?

in python: https://docs.python.org/3.3/library/stdtypes.html#str.ljust

Yeah but, why do you need your own in-house utils library. Node needs batteries included!

Well, it does encourage code re-use, but the issue is that the benefits of code re-use here are far outweighed by the downsides of just writing it yourself.

There are definitely trade-offs involved when making decisions to inline vs import.

Strictly speaking, you're definitely right. Each dependency is an additional point of failure, but so is each additional line of code you inline.

The benefits of these small modules is that they're very thoroughly tested by the entire community. I'd say for most cases, they will be much more robust than any solution an individual developer can spontaneously whip up.

Of course, these modules don't exist in a vacuum, and infrastructure failures such as this one do pose another point of failure you wouldn't have with inlining code, but I think in this particular case it had more to do with the failure of npm to adhere to package management best practices to not include an unpublish feature.

Wow, you were not kidding. I chose a random package of his [1], and indeed: a grand total of 1 LOC...

This is madness.

[1] https://github.com/sindresorhus/float-equal/blob/master/inde...

There is a practical reason for tiny modules in client-side JS that doesn't exist with Python: page load times. If your base layer is going to have third-party dependencies, they better be tiny and do only and exactly what you need.

That said, lodash core is only 4KB, and lodash gives you the ability to build it with only the categories or exact functions you want, so I don't understand what the purpose of a "micro-lodash" would be.

When using a language where

    isArray(arr)

turns into

    toString.call(arr) == '[object Array]'

...then I guess that's more reasonable than if you use something sane.