Comment by userbinator
11 years ago
Small modules are easy to reason about
Yes they are, in the same way that a book in which every page consists of a single word is easier to understand than one with more content per page.
By focusing on the small-scale complexity to such an extreme, you've managed to make the whole system much harder to understand, and understanding the big picture is vital to things like debugging and making systems which are efficient overall.
IMHO this hyperabstraction and hypermodularisation (I just made these terms up, but I think they should be used more) is a symptom of a community that has mainly abandoned real thought and replaced it with dogmatic cargo-cult adherence to "best practices" which they think will somehow magically make their software awesome if taken to extremes. It's easy to see how advice like "keep functions short" and "don't implement anything yourself" could lead to such absurdity when taken to their logical conclusions. The same mentality with "more OOP is better" is what lead to Enterprise Java.
Related article that explains this phenomenon in more detail: http://countercomplex.blogspot.ca/2014/08/the-resource-leak-...
Perhaps another reason for this is that Javascript is inherently unsafe. By relying on these small rigorously tested libraries they are avoiding the need to test their own code, and thus avoiding basic null check or conversion errors.
Other languages handle this with compilers, may be strongly typed, and/or have features that don't allow nulls. Javascript doesn't exactly have that luxury. So maybe it makes sense in Javascript, where it wouldn't in other languages (though one could argue this is a flaw in the language design).
edit: to be clear I'm not really defending the practice, but trying to give a little perspective.
> By relying on these small rigorously tested libraries they are avoiding the need to test their own code, and thus avoiding basic null check or conversion errors.
In practice, many, if not most, of these one-line modules don't even work correctly, and it is difficult to get people to care to collaborate on fixing them instead of just replacing them with a new one-line module that works slightly better as the concept of complex collaboration to perfect one line of code is so foreign to a generation of developers who would rather just throw code to the world and never look back (an issue made all the worse as tiny bugs in these one-liners can almost seem dangerous to fix when you aren't really sure how they are being used: maybe it actually is better to not fix anything and just have people rely on the replacement better module :/).
Odd, the ones I use all have tests, thousands of downloads, and active bug trackers. If I reinvented the wheel, or copy pasted, I wouldn't get those things.
14 replies →
Case in point: the left-pad package has quadratic complexity.
3 replies →
The equivalent in C would then be replacing
#include <math.h> and "-lm"
with
#include <sin.h> #include <cos.h> #include <tan.h> #include <sinh.h> …
and "-lsin -lcos -ltan -lsinh …"
Which is nuts no matter what language you are coding in.
The difference with C is that with a good linker, functions you don't call are removed from the resulting binary (or are maybe not even in the binary if you are dynamically linking to the c runtime).
With javascript however, if you import a hypothetical "math.js" instead of just "sin.js", "cos.js" or "tan.js", then you'll need to download and evaluate a whole bunch of javascript that you might not need.
I'm not defending javascript, because I dislike it and avoid it where possible, but I can understand some of the reasons why short, specific modules came about.
8 replies →
> #include <sin.h> #include <cos.h> #include <tan.h> #include <sinh.h>
I've written worse - at least those cover multiple variations each (or overloads in C++ for float/double/std::complex?/...)
While I'm not a fan of the enforced java route of 1 file = 1 class, I do trend towards 1 file ~ 1 (main, public) thing - which might be a single function with no overloads. #include <string_left_pad.h>? Better than #include <everything.h>, which I see far too often...
I don't have to figure out which grouping of things my coworkers decided to throw something into if I can just #include by name - whereas aggregation headers more often than not will trigger a full project file-system search.
Unnecessary #include s don't accumulate nearly so much when you don't have to audit 100 functions to see if any of them use features from said header.
I don't trigger a rebuild of everything when the only things that #include stuff actually need it.
Lots of benefits!
> and "-lsin -lcos -ltan -lsinh …"
Or sin.o, cos.o, tan.o, sinh.o... which suddenly sounds a lot more reasonable. I wouldn't bat an eye at string_left_pad.o either.
Sure, I want to use tools to simplify the use of those micro-modules by aggregating them into libraries for convenience. But that's not a knock against micro-modules - just their packaging and tooling. Rewind time back far enough and I would've been manually specifying .o files on the command line and kvetching too...
3 replies →
So have libmath depend on libsin, libcos, libtan and libsinh. People who want the kitchen-sink version can get it. People who want just one specific submodule can depend on that. What's not to like?
15 replies →
Oh, but javascript is dynamically typed, so it doesn't matter if your sin-module works with a different kind of arbitrary-precision decimal number module than your geolocation module, your cosine module or your openg-draw-triangle or opengl-draw-circle modules... /sarcasm
2 replies →
You can get approximate feeling if you get rid of a library and don't use -lm but instead unpack it to separate .o files and link against each.
> basic null check
What's funny to me is that leftpad(null, 6, ' ') outputs " null".
Am I right that len(" null") here is 5, so it's working incorreclly?
5 replies →
I think it's a very valid (and concerning) point. JavaScript is made in way that one fears to implement isPositive or isArray.
> Yes they are, in the same way that a book in which every page consists of a single word is easier to understand than one with more content per page.
A more apt metaphor would be separating a book into many small paragraphs that each serves a single purpose makes the book easier to understand. Regardless, the book metaphor misses a lot of the nuance.
Of course taking the approach to an extreme would be detrimental. However, a vast majority of these small modules that are actually widely used consist of functions that may seem trivial at first sight, but actually contain a lot of corner cases and special considerations that a naive inline implementation could miss.
Sure, there are also plenty of trivial one-line modules on npm that don't fit into a such a description, but those are simply side effects of the unprecedented popularity of the platform and its completely open nature, and shouldn't be used to infer any kind of general trend towards "hypermodularisation" because very few reasonable developers would ever import them into their projects.
> A more apt metaphor would be separating a book into many small paragraphs that each serves a single purpose makes the book easier to understand
No, that would not be an apt metaphor for the problem he is describing.
> Regardless, the book metaphor misses a lot of the nuance.
Not if you are trying to understand the point he is trying to make.
Is there any cost to creating modules, uploading them to npm and using them in other projects ? Clearly there is, as is shown by the world-wide breakage from yesterday. This is probably only one of such failure modes.
The point is, breaking everything into micro-modules can have benefits and costs. Ultimately, it is an engineering trade-off.
Now, it is possible that npm modules are at the right level of abstraction, even though other module systems in the world are not this granular. If this is due to the special nature of JavaScript then the point must be argued from that perspective.
> No, that would not be an apt metaphor for the problem he is describing.
That's because I don't agree that the problem he is describing exists, or at least not to the degree he's describing, which was full of hyperbole.
> Is there any cost to creating modules, uploading them to npm and using them in other projects ? Clearly there is, as is shown by the world-wide breakage from yesterday. This is probably only one of such failure modes.
This was a failure on npm's side by including a functionality that allows users to trivially remove packages from a package management system that is used by hundreds of other packages, something that most major package management systems have decided was a bad idea.
> The point is, breaking everything into micro-modules can have benefits and costs. Ultimately, it is an engineering trade-off.
Agreed. And I don't think nearly as many people are erring on the extreme side of this tradeoff to the degree that he's describing.
5 replies →
> for the problem he is describing.
Because it is a strawman.
> By focusing on the small-scale complexity to such an extreme, you've managed to make the whole system much harder to understand
Well put. I've noticed a curious blind spot in how people account for complexity: we count the code that 'does' things much more than the code that glues those things together. This distorts our thinking about system complexity. A cost/benefit analysis that doesn't consider all the costs isn't worth much.
An example is when people factor complex code into many small functions, then say it's much simpler because smaller functions are easier to understand. In fact this may or may not be true, and often isn't. To get a good answer you must consider the complexity you've added—in this case, that of the new function declarations and that of the calls to them—not just the complexity you've removed. But it's hard to consider what you don't see. Why is it so easy not to see things like this? I think it's our ideas about programming, especially our unquestioned assumptions.
The complexity of glue code goes from being overlooked to positively invisible when it gets moved into things like configuration files. Those are no longer seen as part of the system at all. But of course they should be.
I just saw this insanity a couple of days ago:
https://youtu.be/l1Efy4RB_kw?t=343
He refactored a Java application into 1400 classes!
And to add to the mess, until npm introduced deduplication every dependency recursively included it's own dependencies as sub folders, so you get a bajillion tiny dependencies over and over (and sometimes with different versions to boot).
Frankly some of the comments I'm seeing really do reinforce his point: people really don't know how to program. I suspect this is it the flipside of Javascript being popular and accessible - people can churn out products without really knowing what they are doing...
Sorry for the nitpick, but "hyperabstraction"?
left-pad isn't even slightly abstract.
It's not.
The hyperabstraction is in how even tiny functions like this, isPositive, isArray, etc. are being abstracted and turned into individual modules.
Again, maybe I'm nitpicking, but turning something into modules doesn't seem like abstraction... you could call it encapsulation, or modularisation, or maybe extraction.
But abstraction? I don't see how that word is connected to what's happening. Maybe you can explain it.