Comment by atjoslin
11 years ago
Counter-argument:
A good micro-module removes complexity. It has one simple purpose, is tested, and you can read the code yourself in less than 30 seconds to know what's happening.
Take left-pad, for example. Super simple function, 1 minute to write, right? Yes.
But check out this PR that fixes an edge case: https://github.com/azer/left-pad/pull/1
The fact of the matter is: every line of code I write myself is a commitment: more to keep in mind, more to test, more to worry about.
If I can read left-pad's code in 30 seconds, know it's more likely to handle edge cases, and not have to write it myself, I'm happy.
The fault in this left-pad drama is not "people using micro-modules". The fault is in npm itself: all of this drama happened only because npm is mutable. We should focus on fixing that.
> every line of code I write myself is a commitment
That's true. However:
Every dependency you add to your project is also a commitment.
When you add a dependency, you're committing to deal with the fallout if the library you're pulling in gets stale, or gets taken over by an incompetent dev, or conflicts with something else you're using, or just plain disappears. If you add a dependency for just a few lines of code, you're making a way bigger commitment than if you'd just copy/pasted the code and maintained it yourself. That's why so many people are shaking our heads at a 17-line dependency. It's way more risk than it's worth. If you need a better stdlib for your language (some of us write PHP and feel your pain) then find one library that fills in the gaps and use that.
> If you add a dependency for just a few lines of code, you're making a way bigger commitment than if you'd just copy/pasted the code and maintained it yourself.
This is a problem with NPM, not with dependencies. With different package management systems with stable builds and lockfiles, then you pin to a specific version and there is no way upstream can cause problems. A lockfile is a pure win over vendoring.
Yes, there is. Just like NPM's left-pad case. The owner of the package remove the package from the repository. It doesn't matter if you pin to any version if there is no longer a code to download.
The only way to prevent this is to have your own local server for third party package repository.
3 replies →
You're talking about the purely technical aspects of package management that keep your build from breaking. My point is that there's a lot more to it than that. Lockfiles do not keep software from requiring maintenance. Introducing a dependency means handing off a chunk of your software project to someone else to maintain. If they do it wrong, you're on the hook for it.
For example, I was a maintainer for an admin UI component of the last version of Drupal core, and we decided to pull in a query string handling library written by Ben Alman. It was a good decision, and it hasn't caused any problems. But it still meant we were trusting him to maintain part of our codebase. It was also an implicit commitment to every user of Drupal that if Ben quit maintaining that library, we would step in and fix any problems that came up. You don't get rid of that commitment with a lockfile.
2 replies →
OR: you depend on a specific version of the library, that you know that works, and you have none of those problems.
You can't escape problems by bundling specific library versions. You just get a different set of problems. When you require a specific version of a library, you're making your code incompatible with anything that requires a higher or lower version of that library. You're also assuming there will never be a security fix that requires you to update your dependency.
13 replies →
I'll admit I'm at least a little tarnished in my practices due to time spent in enterprises where external dependencies require 6 manager sign offs and a security team exemption, but if this were the case that you didnt want updates to the package, just that one version that worked -
If its just a few lines of code, just copy the thing into your code base? throw a comment in saying "came from xxxx" so anyone reading your code knows that it might look like a overly generic function because it is.
...and then the publisher pulls their library off npm, and another shows up and drops one of the same name in its place, with compatible version numbers (by happenstance or otherwise).
4 replies →
Maintaining a dependency on a library should be much less effort than maintaining 17 lines of code. If it isn't that's a deficiency in your dependency infrastructure.
If you have 100 dependencies, then that's a 100 projects you need to follow and understand the updates for. The minute your dependencies bring in their own dependencies then you start having troubles keep up or even keeping track of the updates. The 17 lines of code you pull in is in most cases a one time deal, having it in a third party library means that you need to keep track of that library for ever.
Honestly, and this is maybe just me being biased against JavaScript, then this is what happens when you pick a language fully knowing it's limited in all sort of silly ways and attempt to use it as a general purpose language. It's not that you can't, but if you need say typing that can tell you if something is an array, then maybe picking JavaScript to begin with wasn't the brightest idea.
There's a ton of libraries and hacks out there, all attempting to fix the fact that JavaScript isn't really good general purpose language. ES6 is fixing a lot of these thing, but it's a little late in the game.
I wouldn't mind so much if these micro-modules were written in a style of thoroughness; heavily commented, heavily documented with pre-conditions, post-conditions and all imaginable inputs and outputs explicitly anticipated and formally reasoned about. I don't mind over-engineering when it comes to quality assurance.
Looking at that left-pad module though - no comments, abbreviated variable names, no documentation except a readme listing the minimally intended usage examples. This is not good enough, in my opinion, to upload to a public repository with the objective that other people will use it. It is indistinguishable from something one could throw up in a couple of minutes; I certainly have no reason to believe that the future evolution of this code will conform to any "expectation" or honour any "commitment" that I might have hopefully ascribed to it.
[EDIT: I've just noticed that there are a handful of tests as well. I wouldn't exactly call it "well tested", as said elsewhere in this thread, but it's still more than I gave it credit for. Hopefully my general point still stands.]
The benefits of reusing other people's code, to a code reuser, are supposed to be something like:
(a) It'll increase the quality of my program to reuse this code - the writer already hardened and polished this function to a greater extent than I would be bothered to do myself if I tried right now
(b) It'll save me time to reuse this code - with the support of appropriate documentation, I shouldn't need to read the code myself, yet still be able to use it correctly and safely.
Neither of those things are true for this module. It's not that the module is small, it's that it is bad.
(True that npm's mutability is a problem too - this is just a side-track.)
Completely agree here - the problem isn't micro-modules. It's partly just a lacking standard library for javascript and largely just exposing issues in npm that the community was pretty ignorant of until just now.
The whole "amaga, it's a whole package for just ten lines of code" is just elitism. Given the number of downloads on things like left-pad, it's clearly useful code.
Agreed as well. In fact, I would posit that this wasn't even really a problem until npm@3 came out and made installing dependencies far, far slower. Yet it was necessary; a standard project using babel + webpack installs nearly 300MB (!!!) of dependencies under npm@2, and about 120MB under npm@3. Both are unacceptable, but at least npm3 helps.
I want to plug ied (https://github.com/alexanderGugel/ied) here, which both installs about 5x faster than npm@2, yet deduplicates as well as npm@3.
No, no, no
Left padding is (almost in all languages) built-in, even C can do it with printf (edited)
The problem is not having a library that offers that, but having this micro-module thing as a whole NPM module. No other language does that.
If it was inside a string helpers, that's great.
But don't make me one single module for just left-padding (or is-integer-number)
I think there's two points related to that:
1) JS is unique in that it is delivered over the wire, so there is a benefit in having micro-modules instead of a bigger "string helpers" module. Things like webpack are changing that now (you can require lodash, and use only lodash.padStart).
2) JS's "standard" library is so small, because it's the intersection of all of the browser implementations of JS dating as far back as you care to support. As pointed out in sibling, a proposal for padLeft is included in ECMA2016. But we'll still need Left-Pad for years after it's adopted.
Point 1 was addressed years ago by Google Closure Compiler, which used "dead code elimination".
Also, the Haxe language, which compiles to JS, has DCE.
Micro-modules is certainly a solution if you don't want to use pre-processing or compilers. So is copy/pasting, or manually writing your own util libs, which seems safer than relying on third parties to provide one-liners for you.
6 replies →
> 1) JS is unique in that it is delivered over the wire, so there is a benefit in having micro-modules instead of a bigger "string helpers" module. Things like webpack are changing that now (you can require lodash, and use only lodash.padStart).
JS isn't even remotely unique in this regard, almost every static language has had dead code removal for decades.
1 reply →
Which is why it's so unsuited for writing non in-browser applications. JS was made for a purpose. We hacked it and discovered (or at least made popular) the benefit of having a fully async STD lib, of using callbacks to manage incoming connection and so on.
The wise course of action would be to take those very good ideas and bake them in languages designed for web/system programming, and keep improving JS for in-browser tasks.
1 reply →
> 1) JS is unique in that it is delivered over the wire
Oh, that's unique? What are these magical languages where code is solely delivered via sneakernet or snailmail?
JavaScript much to my dismay has no sprintf. It's one of the most annoying omissions ever for server side work.
The biggest thing I've run into is that there's no built-in `mkdir -p`. There is, of course, a module that is literally just mkdirp.
require('util').format gets you pretty close.
1 reply →
es6 has one
1 reply →
Uh, no Left Padding is NOT built-in in JavaScript. The proposal to add `String.prototype.padLeft()` was just added to ECMAScript 2016.
JavaScript had a very minimal standard library, it's pretty asinine of you to compare it to C or any other language with a pretty extensive standard library.
C didn't get a standard (or a standard library) until 1989. It had been around for 17 years at that point. Two years after its invention, JavaScript was standardized in 1997. That's almost twenty years ago.
I've never thought of C as being a particularly good example of "extensive standard library."
I wanted to say that it's built-in in pretty much every other language apart from JS
1 reply →
Another interesting argument for micro-modules:
https://github.com/sindresorhus/ama/issues/10#issuecomment-1...
Wow, I feel like I could have written this. Back when I used Python, I had a folder full of functions I would copy-paste between my projects. (And maybe some of the projects contained unit-tests for the function. I didn't always keep those tests in sync.) Updating them was a pain because inevitably each one would get slightly modified over time in each project separately. Eventually, I bundled all of them into a bundle of completely unrelated utility functions in a folder on my computer somewhere, and I would import the folder with an absolute path. Sharing the code I wrote was a pain because of how much it referenced files on my local computer outside of the project. I never considered publishing my utility module because all of the stuff was completely unrelated. I'd rather publish nothing than a horrifying random amalgram that no single project of mine was even related to all of the subject matter present in it.
With npm and the popularity of small modules, it was obvious that I could just cheaply publish each of my utility functions as separate modules. Some of them are about a few dozen lines, but have hundreds of lines of tests and have had significant bugfixes that I am very happy that I haven't had to manually port to dozens of projects. I don't miss copy-pasting code across projects, no matter how many claim I've "forgotten how to program".
I feel exactly the same.
There is something about JavaScript that makes people go a little crazy both for and against it.
I've never seen so many programmers advocate copy/pasting code before...
But regardless of how many insults get thrown around, or how many people seem to think JS is useless or that it's a horrible language, its probably my favorite (and I've done professional work in non-trivial applications from C and C++, to Java and go, to python, Ruby, and PHP to BusinessBasic and even some lisp).
I'm going to keep writing stuff in JS, and I'm going to keep loving it. Regardless of how many people are telling me I'm wrong.
13 replies →
My dependency strategy over time has moved towards more static, project-owns-everything behavior, and specifically "one synchronization script per dependency" - including my own utilities library. The script leaves a echo of the timestamp so that I can also see when it was updated.
That way, different projects can have different versions of everything, and system environment is only important to the synchronization step - trivial to fix up if needed, trivial to copy between machines.
What I see is that a module has a non-zero overhead in complexity in itself. That is, ten 10 line modules and twenty 5 line modules do not yield the same complexity. The modules themselves have a complexity overhead associated, and submodules have their own complexity overhead associated, albeit smaller than first party modules. That complexity is easily seen from the recent situation of unpublishing modules, which resulting in modules multiple steps removed having problems building.
So, when I read "It doesn't matter if the module is one line or hundreds." I call bullshit. There is overhead, it's usually just fairly small (at may event begin to rival the gains from using a module at that level), but that small amount adds up. Once you've had to deal with a dependency graph that's 10 levels deep and contains hundreds or thousands of modules, that small extra complexity imposed by using a module is no longer in total, and comes at a real cost, as we've just seen.
Other module ecosystems have gone through some of the same problems. There was a movement in Perl/CPAN a few years back to supply smaller, more tightly focused modules a while back, to combat the sprawling dependencies that were popping up. The module names were generally suffixed with "Tiny"[1] and the goals where multiple:
- Where possible, clean up APIs where consensus had generally been built over what the most convenient usage idioms were.
- Try to eliminate or reduce non-core dependencies where possible.
- Try to keep the modules themselves and their scope fairly small.
- Remove features in comparison to the "everything included" competitor modules.
This has yielded quite a few very useful and strong modules that are commonly includes in any project. They aren't always tiny, but they attack their problem space efficiently and concisely. Even so, I'm not sure there's ever a module that's a single line of code (or less than 10, given the required statements to namespace, etc), as the point is to serve a problem, not an action.
1: https://metacpan.org/search?size=20&q=%3A%3Atiny&search_type...
I'd say you're playing fast and loose with the term "edge case" there. I call that a bugfix, for a feature that does not work.
It doesn't handle edge cases, it doesn't perform well and it isn't well tested. There is also no documentation. Obviously 30 seconds wasn't enough for you to verify anything at all about this module (namely that it's complete garbage).
And just because some random guy didn't get something as trivial as this right the first time, doesn't mean nobody else can. Also the de facto standard library lodash already has padding utilities, made by people who have a proven track record.
I don't agree with the explosion of micro-modules. There's a reason the vast majority of languages doesn't have them, at least not at function level.
IMO in the Javascript world they're only there in order to minimize script size for front end work. See lodash & lodash-something1 / lodash-something2 / ..., where there's an option of using the whole module or just including 1-function long scripts, precisely to avoid the script size issue.
Is there a solution for this? I know that the Google Closure compiler can remove dead code, ergo making inclusion of large modules less costly in terms of code size. Am I missing some ES6 feature that also helps with this?
You're just trading in the complexity of the code you'd have to write for the delayed complexity of dealing with dependency issues down the line. It's a waste of a trade off for tiny things like this.
Padding a string is a one liner. I'm perfectly happy to write it, test it, convince myself that it works, and then forget about it and just use it.
Completely agree.
Combining this with isomorphic code, cross-browser development and micro-services.
I even wrote a blog post couple of days ago about it [1].
1 : http://www.drinchev.com/blog/increase-your-dependencies/
I agree with the points you've made, but I would also posit that adding a dependency using this mutable package manager is making a commitment to maintain the integrity of that dependency, which is arguably more work than maintaining the handful of lines of code.
What's wrong with copy/paste?
Why do you need an external dependency on something so small?