Comment by runin2k1

Perhaps another reason for this is that Javascript is inherently unsafe. By relying on these small rigorously tested libraries they are avoiding the need to test their own code, and thus avoiding basic null check or conversion errors.

Other languages handle this with compilers, may be strongly typed, and/or have features that don't allow nulls. Javascript doesn't exactly have that luxury. So maybe it makes sense in Javascript, where it wouldn't in other languages (though one could argue this is a flaw in the language design).

edit: to be clear I'm not really defending the practice, but trying to give a little perspective.

> Yes they are, in the same way that a book in which every page consists of a single word is easier to understand than one with more content per page.

A more apt metaphor would be separating a book into many small paragraphs that each serves a single purpose makes the book easier to understand. Regardless, the book metaphor misses a lot of the nuance.

Of course taking the approach to an extreme would be detrimental. However, a vast majority of these small modules that are actually widely used consist of functions that may seem trivial at first sight, but actually contain a lot of corner cases and special considerations that a naive inline implementation could miss.

Sure, there are also plenty of trivial one-line modules on npm that don't fit into a such a description, but those are simply side effects of the unprecedented popularity of the platform and its completely open nature, and shouldn't be used to infer any kind of general trend towards "hypermodularisation" because very few reasonable developers would ever import them into their projects.

> By focusing on the small-scale complexity to such an extreme, you've managed to make the whole system much harder to understand

Well put. I've noticed a curious blind spot in how people account for complexity: we count the code that 'does' things much more than the code that glues those things together. This distorts our thinking about system complexity. A cost/benefit analysis that doesn't consider all the costs isn't worth much.

An example is when people factor complex code into many small functions, then say it's much simpler because smaller functions are easier to understand. In fact this may or may not be true, and often isn't. To get a good answer you must consider the complexity you've added—in this case, that of the new function declarations and that of the calls to them—not just the complexity you've removed. But it's hard to consider what you don't see. Why is it so easy not to see things like this? I think it's our ideas about programming, especially our unquestioned assumptions.

The complexity of glue code goes from being overlooked to positively invisible when it gets moved into things like configuration files. Those are no longer seen as part of the system at all. But of course they should be.

And to add to the mess, until npm introduced deduplication every dependency recursively included it's own dependencies as sub folders, so you get a bajillion tiny dependencies over and over (and sometimes with different versions to boot).

Frankly some of the comments I'm seeing really do reinforce his point: people really don't know how to program. I suspect this is it the flipside of Javascript being popular and accessible - people can churn out products without really knowing what they are doing...

Sorry for the nitpick, but "hyperabstraction"?

left-pad isn't even slightly abstract.

JS does not have an Integer type, so (x > 0) does not work. Now if you add all those checks necessary to see whether you're actually dealing with an integer here, you get to a point where you cannot be sure anymore whether there aren't any weird type coercion issues there and need to start writing tests. Suddenly your whole positive integer check took 1h to write and you still cannot be sure whether there's something you didn't think of. I'd rather use an external library for that that's properly unit tested.

Not defending javascript because I dislike it immensely, but with javascript being a dynamically typed language, and with the way it handles implicit conversions between types, and with trickiness around NaN, Infinity and null objects, there are sufficient edge cases that writing a correct function to test if some variable contains a positive integer is not as trivial as it is with more sane languages, and is likely to trip up even experienced developers if they miss one or more of those edge cases.

Having pre-written modules that account for all edge cases that have been battle-tested by hundreds, thousands, or even millions of users does have merit.

The main problem here is with the Javascript language, which makes code evolve to things like 'is-positive-integer'.

If you want to build is-bigger-than-5 yourself, I recommend pulling in fivejs[1] as a dependency.

[1] https://github.com/jackdcrawford/five

add-1-to-a-number

This seems to be a very common example in educational materials about how to create a function in some programming language. Perhaps they should all be prefaced with "NOTE: you should not actually make such a function, because its triviality means that every time you use it you will be increasing the complexity of your code", or a longer discussion on when abstraction in general is appropriate and when it is not.

Ditto for any other abstraction --- I've found there's too much of an "abstraction is good so use it whenever you can" theme in a lot of programming/software design books and not enough "why abstraction can be bad" sort of discussion, which then leads to the "you can never have too much" mentality and the resultant explosion of monstrous complexity. Apparently, trusting the reader to think about the negatives and exercise discretion didn't work so well...

It doesn't.

Of course we have a number of packages that are truly trivial and should probably be inlined. Npm is a completely open platform. Anyone can upload any module they feel like. That doesn't mean anyone else will feel the need to use them.

I think you'll find that the vast majority of small modules that are widely used are ones that also cover obscure corner cases and do benefit from widespread use and testing.

No, you see "is-bigger-than-5" clearly does 4 different things. You need these modules:

module.exports = exec("===");

module.exports = bigger(a, b){return a > b};

module.exports = () => return "than";

module.exports = (a) => a.search('ABCD...');

.. or something like that. These are all plugins of course.

Do you seriously think that "word starts with a capital letter" is an easy function to write?

I feel like you haven't spent enough time with Unicode.

The three cases you state aren't nearly as hairy as determining whether something is a positive integer.

Someone might do something like ~~n to truncate a float. That works fine until you go above 32 bits. Math.trunc should be used instead. Someone might do something like n >>> 0 === n, and using any bitwise operators will always bake in the assumption that the number is 32 bits. Do you treat negative 0 as a negative integer? Detecting negative 0 is hairy. So, to avoid bad patterns, it makes sense.

For is-bigger-than-5(x), x > 5 is not hairy. For add-1-to-a-number(n), n + 1 is not hairy.

For word-starts-with-capital-letter(word)? That one is actually pretty hairy. There are programmers that would write a regular expression /^[A-Z]/, or check the charCode of the first character being within the A-Z range, amongst other solutions. An appropriate solution, however, would be (typeof word == "string" && word.length && word[0] == word[0].toUpperCase() && word[0].toUpperCase() != word[0].toLowerCase()), because the toUpperCase and toLowerCase methods are unicode-aware, and presently you can't access unicode properties using Javascript's flavor of regular expressions.

People have started to point out the existence of modules in this very vein. And, of course, write them. With two such together, you can probably get is-maybe-5 .

* https://news.ycombinator.com/item?id=11359302

If those functions are reusable or complex enough (partly due to deficiencies in javascript the language, sure) then why not?

>add-1-to-a-number

i++

The isArray package has 72 dependent NPM packages, it's certainly not undiscoverable. The leftpad package gets 5000 downloads a month, that's quite a bit of testing for edge cases, compared to the none that I would have gotten had I implemented this myself.

Intuitively, this thread wouldn't exist if your assertion were correct.

Discoverability though is... poor seems like an argument for improving discoverability, not against having small modules.

Can we ever achieve this?

We can't exactly search the code to find out what it does, otherwise you'd basically be reading all the code to discover it's true behavior, which negates the usefulness of modules in the first place...

Any search will have to rely on developer-made documentation, and/or meta data. This is great in theory, but documentation is rarely well maintained, and/or someone changes the code but neglects to update the documentation.

This leaves us with the situation we have today. A search that somewhat works kindof, and mostly you rely on googling the feature you need and looking for what seems to be the most popular choice.

I'm not sure how this situation can be made better, especially if we continue down a path of having all these "one-liner" libraries/modules that anyone can make and stick out there into the unsuspecting world. When I need a square root function, and my search returns 800 one-liner modules, how am I supposed to pick one that I know is reasonably well done, and does what it says it will do, without looking under the hood - you'll end up just picking whatever seems to be popular...

You're totally right. Most developers don't develop this way, and that makes the benefits of this approach far less pronounced than they would be if they did. It also doesn't negate the benefits entirely, of course.

Another benefit of small well understood components is that they are easier to write tests for. Do these npm modules have good test coverage? Did the leftpad module have a unit test that would have caught the issue?

I think the gist of this whole discussion ( at least the OMG WHY?!?! ) part, can be easily explained by an excerpt from your example comment that sums up in a nutshell the all too pervasive mindset I've seen over the years:

"...LOC is pretty much irrelevant. It doesn't matter if the module is one line or hundreds. It's all about containing complexity. Think of node modules as lego blocks. You don't necessarily care about the details of how it's made. All you need to know is how to use the lego blocks to build your lego castle. By making small focused modules you can easily build large complex systems without having to know every single detail of how everything works..."

By LOC he's referencing 'ye ole Lines of Code paradigm, and trying to make the point that in the end it just doesn't measure up to the prime directive of "containing complexity".

... and that's where I beg to differ.

What I think is being completely overlooked here is the net cost of abstracting away all that complexity... It's performance.

Every extra line of code, extraneous function call, unnecessary abstraction ( I'm looking at you promise ), every lazy inclusion of an entire library for the use of a shortcut to document.getElementById -- these all add unnecessary costs from the network down to the cpu.

And who pays these costs?

The end users of your slow, memory hogging, less-than-ideal set of instructions to the cpu that you call an application... but hey, it's easier for you, so there's that.

> Maybe some people have so little to do that they can author or find a relevant package for every single ~10 line function they need to use in their code and then spend countless commits bumping project-versions and updating package.json files. I have no idea how they get work done though...

Rather than copy paste someone else's unmaintained thing and handle all the bugs, or write their own code and unit tests, they use the same library as a few thousand other people?

How is it superior? Simple.

How do I systematically make sure that I have the latest version of every stackoverflow code snippet? If it's a new post, it may not have all the edge cases fixed yet. So now I have to check back on each of the X number of snippets I've copied.

In the npm approach, I can easily tell if there's a new version. For prod, I can lock to a specific version, but in my test environment, I can use ^ to get newer versions and test those before I put them in production.

If the edge case of new version of a package breaks my code, I've learned that I'm missing a unit test. Plus, the question isn't whether this bad thing might happen on occasion, the question is whether this approach is, on balance, superior to cutting and pasting random code snippets into my code. I think the downside of the npm approach is less than the downside of the copypasting from stackoverflow approach.

And every moderately useful npm package I've looked at has very good to great documenation.

With the advent of ES6 modules's selective imports and tree-shaking, that's definitely quickly becoming a better approach. With the old CommonJS modules, you need to be concerned about overall code size, which is where small, single purpose modules excel, and why this approach has proliferated to this degree.

People need modules like leftpad to tick the "maintains moderately popular open source project" checkbox. Instant hirability.

I don't want to claim that it would be a directly calculated career move, more like starting a blog: you admire a good blog, you want to be more like the blogger, you start your own one. On the dim chance that it will become both good and not abandoned after the third post. Nanomodules can be just like that, the air guitar of would-be rockstar coders. This is certainly not the only reason for their existence (and even the air-guitar aspect has a positive: low barrier of entry that might lead to more serious contribution), but the discussion would be incomplete if it was ignored.

if leftpad were in the node stdlib, would you still be making the same point about knowing computer science and using leftpad?

in python: https://docs.python.org/3.3/library/stdtypes.html#str.ljust

Yeah but, why do you need your own in-house utils library. Node needs batteries included!

Well, it does encourage code re-use, but the issue is that the benefits of code re-use here are far outweighed by the downsides of just writing it yourself.

There are definitely trade-offs involved when making decisions to inline vs import.

Strictly speaking, you're definitely right. Each dependency is an additional point of failure, but so is each additional line of code you inline.

The benefits of these small modules is that they're very thoroughly tested by the entire community. I'd say for most cases, they will be much more robust than any solution an individual developer can spontaneously whip up.

Of course, these modules don't exist in a vacuum, and infrastructure failures such as this one do pose another point of failure you wouldn't have with inlining code, but I think in this particular case it had more to do with the failure of npm to adhere to package management best practices to not include an unpublish feature.

Wow, you were not kidding. I chose a random package of his [1], and indeed: a grand total of 1 LOC...

This is madness.

[1] https://github.com/sindresorhus/float-equal/blob/master/inde...

There is a practical reason for tiny modules in client-side JS that doesn't exist with Python: page load times. If your base layer is going to have third-party dependencies, they better be tiny and do only and exactly what you need.

That said, lodash core is only 4KB, and lodash gives you the ability to build it with only the categories or exact functions you want, so I don't understand what the purpose of a "micro-lodash" would be.

When using a language where

    isArray(arr)

turns into

    toString.call(arr) == '[object Array]'

...then I guess that's more reasonable than if you use something sane.

Of course that's why we write functions. But we can't share a function with others and across projects... so we make it into a module to do that.

It's like (micro)crowdsourcing (the smallest components of) the standard library that JavaScript never had.

Some bit of logic could go from being DRY in one project, to DRY in all of my projects, ... to eventually be DRY in all projects. It's globally DRY.

>hasn't reduced the number of paths to test

But it has: it's now tested, and you don't need to write tests for it next time you want to use it.

>100% test coverage is rarely a good thing

Not sure what your argument is here. Sure, it may not be helpful but are you saying that one should strive for less than 100% coverage, because "it's rarely good"?

> 100% test coverage is rarely a good thing

Do you mean rarely useful, or actively harmful?

See:

  isPositive = (x) => +x === x && x > 0

In which conditions does it return a wrong value? I haven't found any.

Not to mention the fact that adding many floating point numbers in a naive way results in a serious error accumulation, which is why things like Kahan summation algorithm[1] exist.

[1] - https://en.wikipedia.org/wiki/Kahan_summation_algorithm

Every once in a while, I come to HN, and inevitably find a post about some new JavaScript technology on the front page, wherein I find some comment that gives me a new appreciation for the web app language I use at work, which fortunately is not JavaScript.

That's not true. In most javascript implementations, integers will be represented by real 32 bit integers.

For now I suggest using lodash functions exported as modules.

For example: https://www.npmjs.com/package/lodash.mean

Or, in light of `left-pad`: https://www.npmjs.com/package/lodash.padstart

You get the benefit of a "small, focused" module but still rely on a large and stable project with a great maintainer that cares about performance and testability.

There's already `Array.prototype.reduce()` in most browsers, which will do summing, string concatenation, and any other kind of aggregation:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

If you need something more convenient I highly recommend just adding lodash and getting a whole bag of rock-solid awesome tools.

here is the source for that npm package

https://github.com/bytespider/average/blob/master/src/averag...

and here is the code copied out in full

module.exports = function average(values) { for (var i = 0, sum = 0; i < values.length; ++i) { sum += values[i]; } return sum / values.length; };

is this worth adding a dependency to your build for?

Well if I was interviewing the guy, my second or third question would be "tell me about these packages you published on NPM"

It's going to be damn hard to make an average function sound impressive.

That plus perhaps some kind of dopamine kick these people get out of this whole charade... Sindre Sorhus keeps describing himself as an "Open Source addict", after all.

> I do expect to be able to call "average" on a list of numbers without writing it myself

Just out of interest, what kinds of functions would you expect to have to write yourself, if you're not happy about calculating the average of a list of numbers?

It should, but the ECMAScript spec doesn't have it, and Node.js team prefers to keep the core libraries to a minimum and definitely do not want to modify prototypes. Thus we depend on userland to provide those features for us.

That could be one dependent module that got installed in 54 places. More likely, it was mostly various bots that install everything on some semi-regular schedule.

Let's say these are just prescient trolls. For our sanity.

npm has namespaces. https://docs.npmjs.com/misc/scope

Don't be .... Mean.

You bring up a good point with respect to copyright/licensing --- at what point does code stop being copyrightable and become public domain simply due to pure triviality? There are not very many sane ways to write an averaging function, and the most straightforward one (add up all the value, then divide) would probably be present in tons of other codebases.

Use the null object pattern.

If you're checking a value to see if it's set by testing for null/undefined, you're doing it wrong.

This is good advice for any language, not just JS.

Besides, using null as the default for an undefined value is a mistake. Null should indicate a non-value not the absence of a value. Maybe one day the static OOP languages will die so devs have a chance to learn the difference.

npm does a decent enough job deduping shared packages. The reason is different packages consume different versions of things. Java only lets you have a single version of a package. If the API changes, it can be a pain when your dependencies were written for different versions. That slows adoption since you have to wait until everyone you depend on has updated first.

its a long standing issue, and gets worse when there is node_modules inside node_modules inside node_modules ... so on that someone suggested renaming node_modules to n_m . https://github.com/nodejs/node-v0.x-archive/issues/6960#issu...

It could have been a simple for loop with a break whenever one evaluates false. Blazing fast in every javascript engine, and easy to see all the code in one place.

Instead, it’s a slow tower of bloat, which you need to read 5 files to reason about.

Javascript implementation:

    function passAll() {
      var fns = [].slice.call(arguments);
      return function() {
        for (var i = 0, len = fns.length; i < len; i++) {
          if (!fns[i].apply(null, arguments)) { return false; }
        }
        return true;
      };
    }

Or Coffeescript (including the function check):

    passAll = (fns...) ->
        for fn in fns then if typeof fn isnt 'function'
            throw new TypeError 'all funcs should be functions'
        (args...) ->
            (return false if not fn args...) for fn in fns
            true

If it were being used only in places in the code where it was rarely called, it would be fine to create a handful of extra functions, make a few dozen function calls, etc. In my opinion, anything that aspires to be a library of basic infrastructure type code should try to have efficient and simple code at the expense of being slightly syntactically longer than the highly abstracted version.

Because if people start using it for such things as an "is positive integer" check (Which, to be fair, they should not be doing. Nobody should be constructing 2 new javascript functions every time they want to check if an object is a positive integer. But apparently they are...), then it could easily make its way into somebody’s inner loops.

The end result of this general culture (not picking on anyone in particular) is that trivial Javascript programs end up having their performance tank, because every tiny bit of useful real work gets weighed down by two orders of magnitude of incidental busywork creating and tearing down abstractions built on towers of other abstractions. Because CPUs aren’t very well optimized for this kind of workload, which involves lots of branching and cache misses, it’s worse still, probably more like 4–5 orders of magnitude slower than optimized code written in a low-level language.

Ultimately every idle facebook page and gmail page and news article in some background browser tab ends up sucking up noticeable amounts of my laptop’s CPU time and battery, even when they’re doing nothing at all for me as the user.

Be that as it may, is-positive doesn't handle any edge cases https://github.com/kevva/is-positive/blob/master/index.js

That's half the problem with these stupid micromodules; they're not abstracting complexity, they're obfuscating simplicity.

> I don't use NPM so maybe it doesn't really matter aside from the level of abstraction being implemented being relatively ridiculous.

Except, again, it's no more ridiculous than pick a random bit of code from anywhere and isn't any more emblematic than that random bit because, again, it's not used anywhere.

> [Quick Edit] Basically I'm saying "Get off my lawn ya kids!"

tl;dr: going for buzzfeed headlines but for comments, got it. Good job, I guess?

That's good, even if nobody is perfectly enough to a botnet. I thought more of the user running the node application.

When every major Node package depends on this, starting with Babel, then it actually does becomes something thats required.

There's no such thing as "don't use it if you don't like it!" in this case.

Javascript is a completely broken, shameful system that rightfully deserves all the scorn it's gotten. It really is the worst.

(note that i actually write Python & VanillaJS, and avoid Node packages entirely for now.)