Comment by bcantrill
10 years ago
The grand irony is that Larry was one of the earliest advocates of open sourcing the operating system at Sun[1] -- and believed that by the time Sun finally collectively figured it out and made it happen (in 2005), it was a decade or more too late.[2] So on the one hand, you can view the story of BitKeeper with respect to open source as almost Greek in its tragic scope: every reason that Larry outlined for "sourceware"[3] for Sun applied just as much to BK as it did to SunOS -- with even the same technologist (Torvalds) leading the open source alternative! And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005, but I also think this represents a forced dichotomy of "winners" and "losers." To the contrary, I would like to believe that the ongoing innovation in the illumos communities (SmartOS, OmniOS, etc.) proves that it's never too late to open source software -- that open source communities (like cities) can be small yet vibrant, serving a critical role to their constituencies. In an alternate universe, might we be running BK on SunOS instead of git on Linux? Sure -- but being able to run an open source BK on an open source illumos is also pretty great; the future of two innovative systems has been assured, even if it took a little longer than everyone might like.
So congratulations to Larry and crew -- and damn, were you ever right in 1993! ;)
[1] Seriously, read this: http://www.landley.net/history/mirror/unix/srcos.html
[2] The citation here is, in that greatest of all academic euphemisms, "Personal communication."
[3] "Sourceware" because [1] predates the term "open source"
Yeah this irony is not lost on me. But in both cases, the companies acted in self interest. Neither had the guts to walk away from their existing revenue stream. It's hard to say what would have happened.
It's been an interesting ride and if nothing else, BK was the inspiration for Git and Hg, that's a contribution to the field. And maybe, just maybe, people will look at the SCCS weave and realize that Tichy pulled the wool over our eyes. SCCS is profoundly better.
Thank you for contributing to the development and evangalizing of DVCS, directly (BK) and indirectly (the ideas and inspiration for git, hg).
It's probably fair to say the DVCS accelerated the growth of the entire software industry.
Was BitKeeper the first version control system to "think distributed" ?
23 replies →
> Thank you for contributing to the development and evangalizing of DVCS, directly (BK) and indirectly (the ideas and inspiration for git, hg).
I concur.
> the companies acted in self interest
Sun, could, at least, make a profit building workstations and servers and licensing chips.
It's actually very sad they don't build those SPARC desktops anymore.
I really doubt it. There's not enough demand to make them competitive in price-performance. The server chips have stayed badass but still niche market. There's even open-source, SPARC HW with reference boards for sell from Gaisler. That didn't take off.
I liked the desktops but there's no money in them. Market always rejects it. So does FOSS despite it being the only open ISA with mainstream, high-performance implementations.
14 replies →
They do, sort of. Intel were using SPARC cores in at least one of their "Management Engine" devices in chipsets recently anyway. ;)
> realize that Tichy pulled the wool over our eyes.
ref: https://en.wikipedia.org/wiki/Revision_Control_System, https://en.wikipedia.org/wiki/Walter_F._Tichy
Yeah, that. I think he got a PhD for RCS and what he should have gotten is shown the door. RCS sucks, SCCS is far, far better.
Just as an example, RCS could have been faster than SCCS if they had stored at the top of the file the offset to where the tip revision starts. You read the first block, then seek past all the stuff you don't need, start reading where the tip is.
But RCS doesn't do that, it reads all the data until it gets to the tip. Which means it is reading as much data as SCCS but only has sorta good performance for the tip. SCCS is more compat and is as fast or faster for any rev.
And BK blows them both away, we lz4 compress everything which means we do less I/O.
RCS sucked but had good marketing. We're here to say that SCCS was a better design.
> in both cases, the companies acted in self interest. Neither had the guts to walk away from their existing revenue stream.
Why does bitmover have the guts now?
he explained the move in another comment https://news.ycombinator.com/item?id=11668492
I though SCCS had the same problems as RCS. What did it do differently?
RCS is patch based, the most recent version is kept in clear text and the previous version is stored as a reverse patch and so on back to the first version. So getting the most recent version could be fast (it isn't) but the farther back you go in history the more time it takes. And branches are even worse, you have to patch backwards to the branch point and then forwards to the tip of the branch.
SCCS is a "weave". The time to get the tip is the same as the time to get the first version or any version. The file format looks like
That's "insert in version 1" data "end of insert for version one".
Now lets say you added another line in version 2:
So how do you get a particular version? You build up the set of versions that are in that version. In version 1, that's just "1", in version 2, that's "1, 2". So if you wanted to get version 1 you sweep through the file and print anything that's in your set. So you print the first line, get to the ^AI 2 and look to see if that's in your set, it isn't, so you skip until you get to the ^AE 2.
So any version is the same time. And that time is fast, the largest file in our source base is slib.c, 18K lines, checks out in 20 milliseconds.
29 replies →
In short, RCS maintains a clean copy of the head revision, and a set of reverse patches to be applied to recreate older revisions. SCCS maintains a sequence of blocks of lines that were added or deleted at the same time, and any revision can be extracted in the same amount of time by scanning the blocks and retaining those that are pertinent.
Really old school revision control systems, like CDC's MODIFY and Cray's clone UPDATE, were kind of like SCCS. Each line (actually card image!) was tagged with the ids of the mods that created and (if no longer active) deleted it.
4 replies →
I've read that "sourceware" article before, in the distant past when it was still a roughly accurate picture of the market (maybe 1995 or 1996). It's weird to read it again now, in a world that is so remarkably changed. Linux, the scrappy little upstart with a million or so users at the time of the paper, is now the most popular OS (or at least kernel) on the planet, powering billions of phones and servers. NT was viewed as the unfortunate but inevitable future of server operating systems.
I remember looking at IT jobs back then, and seeing a business world covered in Windows NT machines; I even got my MCSE (alongside some UNIX certifications that I was more excited about), because of it. Looking at jobs now, the difference is remarkable, to say the least. Nearly every core technology a system administrator needs to know is Open Source and almost certainly running on Linux.
And, the funny thing is that the general prescription (make a great Open Source UNIX) is exactly what it took to save UNIX. It just didn't involve any of the big UNIX vendors in a significant way (the ones spending a gazillion dollars on UNIX development at the time). Linux got better faster than Sun got smarter, and ate everybody's lunch, including Microsoft. Innovator's Dilemma strikes again.
Apple is an interesting blip on the UNIX history radar, too...though, they're likely to lose to the same market forces in the end, as phones become commodities. I'm a bit concerned that it's going to be Android, however, that wins the mobile world since Android is nowhere near the ideal OS from an Open Source and ethical perspective; but, I guess they got the bits right that Larry was suggesting needed to be right.
Anyway, it was a weird flashback to read that article. Things change, and on a scale that seems slow, until you look back on it, and see it's "only" been a couple of decades. In the grand scheme of things, and compared to the motion of technology prior to the 1900, that's a blink of an eye.
Eh. Linux does have a LOT of problems. Well, so does everything, but it's not like it's "great." More like "good."
But yeah, it was weird that everybody thought NT was going to be the future. And now, MS has opensourced a good deal of infrastructure, is working with Node, has announced an integrated POSIX environment for Windows. And since it's in corporate, it might even be able to fix the fork(2) performance problems.
Great is a relative term. But, can you name an OS, especially a UNIX, that is better in the general case? By "general case", I mean, good for just about anything, even if there's something better for some niche or role. Also, take into account the world we live in: More computing happens on servers and phones than on desktops and laptops; and judge the OS based on how it's doing in those roles.
I sincerely consider Linux a great UNIX. Probably the best UNIX that's existed, thus far. There are warts, sure. Technically, Solaris had (and still has, in IllumOS and SmartOS) a small handful of superior features and capabilities (at this point one can list them on one hand, and one could also list some "almost-there" similar techs on Linux). But, I assume you've used Solaris (or some other commercial UNIX) enough to have an opinion...can you honestly say you enjoyed working on it more than Linux? The userspace on Solaris was always drastically worse than Linux, unless you installed a ton of GNU utilities, a better desktop, etc. But, Linux brought us a UNIX we could realistically use anywhere, and at a price anyone could afford. That's a miracle for a kid that grew up lusting after an Amiga 3000UX (because it was the closest thing to an SGI Indy I could imagine being able to afford).
22 replies →
"I'm a bit concerned that it's going to be Android, however, that wins the mobile world since Android is nowhere near the ideal OS from an Open Source and ethical perspective; but, I guess they got the bits right that Larry was suggesting needed to be right."
They've already won; Apple isn't coming back (did they "go thermonuclear" in the end or did that nonsense die with Mr Magical Thinking?).
Don't confuse Android with Google; you can grab the source and do what you want with it, like Cyanogenmod have, or like millions of hobbyists are doing themselves. It's all available with bog standard open source licenses - no need to worry about ethics.
Not really. AOSP died when Jean-Baptiste Queru quit in protest out of AOSP not really being open source. He works at Yahoo now.
http://www.engadget.com/2013/08/07/aosp-maintenance-head-lea...
1 reply →
This is a bit funny because Mercurial is partly named after Larry.
https://groups.google.com/d/msg/mercurial_general/c3_SM3p7S1...
Just for the record this thread is where I learned that. And yup, it sorta fits. For better or worse.
The point about small communities is great. I've never seen it spelled out like that, but it captures what I was thinking perfectly. People get caught up on popularity as the only meaningful metric for success, but it really isn't.
> And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005,
In case of software, it is never too late: as you once put it so eloquently, software does not suddenly stop working and does not have an expiration date.
If this software works and works well, then Paul Graham's revolutionary idea of when you choose technology, you have to ignore what other people are doing, and consider only what will work the best applies. (Common sense really, but apparently not to the rest of our industry.)
If this software will work the best, and do exactly what I want and need it to do, I have enough experience to know not to care that everyone else runs something like git just because that is trendy right now. (A lesson appreciated by those who run SmartOS because it is the best available technology for virtualization, cloud, and performance, instead of running Linux and Docker.)
> software [...] does not have an expiration date.
xscreensaver does. (-:
* https://news.ycombinator.com/item?id=11412081
Sun is also no more... and it's not at all clear they would've survived had Solaris been open sourced a decade earlier.
Yes, you're absolutely right that there are a ton of startups built on opensolaris (who have proprietary code they haven't and don't intend to ever give back to the community), and there is smartos/omnios/illumos as well. But none of those projects would have in any way contributed to the health of Sun Microsystems, nor provided the funding to get Solaris to where it is today. ZFS may have never seen the light of day if Solaris were open sourced in 1995.
> ZFS may have never seen the light of day if Solaris were open sourced in 1995.
It depends on how that would have affected Jeff Bonwick. If it kept him from deciding that Sun ought to develop a new filesystem, promising Matthew Ahrens a job writing one out of college and working together with Matt on it, ZFS would never have existed.
Some history. I was at Sun and Bob Hagmann was teaching at Stanford and got me to be a TA there. He retired and Stanford asked me if I would teach CS240B so I did. Jeff Bonwick was student and I recognized his ability and recruited him to Sun. He said "I have no experience programming in C" and I said "You are smart. I can teach you C, I can't teach you smart".
I also told him that he'd go way farther at Sun than I did and I was right, I think he made DE, I didn't. He played the game better. Smart guy. Him, Bryan, Bill Moore, those guys were the new Sun in my mind.
1 reply →
I actually posted before I saw your comment. I guess I was right. So are you happy to finally have an open source bring-over-modify-merge VCS whose command set makes sense?
Bit keeper is a great example of what happens when you do not open source your code. I have cited it that way many times.
Except that we've been around for 18 years and made payroll without fail that entire time. Supported a team of 10-15 people every year. That's something, many many companies in the valley, including many that open sourced everything, have not done as well.
You may have done more by open sourcing whatever it is that you have done; if so congrats.
My remark was intended to cite how much farther bit keeper could have gone had it been open source from the start rather than belittle what bit keeper accomplished.
At work, many of my newer colleagues have backgrounds in closed source software development. We are developing software that has no exact analog to existing software, and we hope that it will have a big impact. If it becomes as important as we think it could be, then bit keeper vs git is a fantastic example of why our work should be open source from the start.
3 replies →
I have no idea why people are down voting this. In an alternative universe, we would all be using bit keeper. The reason we are not is mainly because Larry McVoy ceded the market to git and mercurial because he was afraid of disrupting his existing business. git and mercurial would never have existed had he practiced at Bit Movers what he preached at Sun.