The grand irony is that Larry was one of the earliest advocates of open sourcing the operating system at Sun[1] -- and believed that by the time Sun finally collectively figured it out and made it happen (in 2005), it was a decade or more too late.[2] So on the one hand, you can view the story of BitKeeper with respect to open source as almost Greek in its tragic scope: every reason that Larry outlined for "sourceware"[3] for Sun applied just as much to BK as it did to SunOS -- with even the same technologist (Torvalds) leading the open source alternative! And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005, but I also think this represents a forced dichotomy of "winners" and "losers." To the contrary, I would like to believe that the ongoing innovation in the illumos communities (SmartOS, OmniOS, etc.) proves that it's never too late to open source software -- that open source communities (like cities) can be small yet vibrant, serving a critical role to their constituencies. In an alternate universe, might we be running BK on SunOS instead of git on Linux? Sure -- but being able to run an open source BK on an open source illumos is also pretty great; the future of two innovative systems has been assured, even if it took a little longer than everyone might like.
So congratulations to Larry and crew -- and damn, were you ever right in 1993! ;)
Yeah this irony is not lost on me. But in both cases, the companies acted in self interest. Neither had the guts to walk away from their existing revenue stream. It's hard to say what would have happened.
It's been an interesting ride and if nothing else, BK was the inspiration for Git and Hg, that's a contribution to the field. And maybe, just maybe, people will look at the SCCS weave and realize that Tichy pulled the wool over our eyes. SCCS is profoundly better.
I've read that "sourceware" article before, in the distant past when it was still a roughly accurate picture of the market (maybe 1995 or 1996). It's weird to read it again now, in a world that is so remarkably changed. Linux, the scrappy little upstart with a million or so users at the time of the paper, is now the most popular OS (or at least kernel) on the planet, powering billions of phones and servers. NT was viewed as the unfortunate but inevitable future of server operating systems.
I remember looking at IT jobs back then, and seeing a business world covered in Windows NT machines; I even got my MCSE (alongside some UNIX certifications that I was more excited about), because of it. Looking at jobs now, the difference is remarkable, to say the least. Nearly every core technology a system administrator needs to know is Open Source and almost certainly running on Linux.
And, the funny thing is that the general prescription (make a great Open Source UNIX) is exactly what it took to save UNIX. It just didn't involve any of the big UNIX vendors in a significant way (the ones spending a gazillion dollars on UNIX development at the time). Linux got better faster than Sun got smarter, and ate everybody's lunch, including Microsoft. Innovator's Dilemma strikes again.
Apple is an interesting blip on the UNIX history radar, too...though, they're likely to lose to the same market forces in the end, as phones become commodities. I'm a bit concerned that it's going to be Android, however, that wins the mobile world since Android is nowhere near the ideal OS from an Open Source and ethical perspective; but, I guess they got the bits right that Larry was suggesting needed to be right.
Anyway, it was a weird flashback to read that article. Things change, and on a scale that seems slow, until you look back on it, and see it's "only" been a couple of decades. In the grand scheme of things, and compared to the motion of technology prior to the 1900, that's a blink of an eye.
Eh. Linux does have a LOT of problems. Well, so does everything, but it's not like it's "great." More like "good."
But yeah, it was weird that everybody thought NT was going to be the future. And now, MS has opensourced a good deal of infrastructure, is working with Node, has announced an integrated POSIX environment for Windows. And since it's in corporate, it might even be able to fix the fork(2) performance problems.
"I'm a bit concerned that it's going to be Android, however, that wins the mobile world since Android is nowhere near the ideal OS from an Open Source and ethical perspective; but, I guess they got the bits right that Larry was suggesting needed to be right."
They've already won; Apple isn't coming back (did they "go thermonuclear" in the end or did that nonsense die with Mr Magical Thinking?).
Don't confuse Android with Google; you can grab the source and do what you want with it, like Cyanogenmod have, or like millions of hobbyists are doing themselves. It's all available with bog standard open source licenses - no need to worry about ethics.
The point about small communities is great. I've never seen it spelled out like that, but it captures what I was thinking perfectly. People get caught up on popularity as the only meaningful metric for success, but it really isn't.
> And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005,
In case of software, it is never too late: as you once put it so eloquently, software does not suddenly stop working and does not have an expiration date.
If this software works and works well, then Paul Graham's revolutionary idea of when you choose technology, you have to ignore what other people are doing, and consider only what will work the best applies. (Common sense really, but apparently not to the rest of our industry.)
If this software will work the best, and do exactly what I want and need it to do, I have enough experience to know not to care that everyone else runs something like git just because that is trendy right now. (A lesson appreciated by those who run SmartOS because it is the best available technology for virtualization, cloud, and performance, instead of running Linux and Docker.)
Sun is also no more... and it's not at all clear they would've survived had Solaris been open sourced a decade earlier.
Yes, you're absolutely right that there are a ton of startups built on opensolaris (who have proprietary code they haven't and don't intend to ever give back to the community), and there is smartos/omnios/illumos as well. But none of those projects would have in any way contributed to the health of Sun Microsystems, nor provided the funding to get Solaris to where it is today. ZFS may have never seen the light of day if Solaris were open sourced in 1995.
> ZFS may have never seen the light of day if Solaris were open sourced in 1995.
It depends on how that would have affected Jeff Bonwick. If it kept him from deciding that Sun ought to develop a new filesystem, promising Matthew Ahrens a job writing one out of college and working together with Matt on it, ZFS would never have existed.
I actually posted before I saw your comment. I guess I was right. So are you happy to finally have an open source bring-over-modify-merge VCS whose command set makes sense?
Except that we've been around for 18 years and made payroll without fail that entire time. Supported a team of 10-15 people every year. That's something, many many companies in the valley, including many that open sourced everything, have not done as well.
You may have done more by open sourcing whatever it is that you have done; if so congrats.
I have no idea why people are down voting this. In an alternative universe, we would all be using bit keeper. The reason we are not is mainly because Larry McVoy ceded the market to git and mercurial because he was afraid of disrupting his existing business. git and mercurial would never have existed had he practiced at Bit Movers what he preached at Sun.
For people who don't know the history -- McVoy offered free bitkeeper licenses to various open source projects, and the Linux kernel switched to it.
After Andrew Tridgell (SAMBA, among other projects) reverse-engineered the bitkeeper protocol [1] in order to create his own client, the license was rescinded for everyone.
It's astonishing to me that Git has won out given how much easier it's been for me to explain Hg to other people than to explain Git. To this day, in our SVN workflow at my company, nontechnical people who have merely seen a Hg diagram on a whiteboard by my desk immediately grasped the idea and the lingo, and ask me questions like "hey, can you branch the code to commit those changes and push them to the testing server? This thing's really cool and we don't mind playing with the alpha version, but we might scrap it all later."
From what I read, he connected to a bit keeper repository via telnet on port 5000, executed the help command and then used that information to write an incomplete client. That does not sound like reverse engineering to me.
It wasn't. He gave conclusive reply which established that it's ethical (just telneting and help). Unless you believe samba and everything else is unethical and you club every reverse engineering under one umbrella, your comment is wrong. http://www.theregister.co.uk/2005/04/14/torvalds_attacks_tri...
I don't think it's fair to call people douches because they are committed to their moral principals. Especially so here, where the benefit to humanity over the alternative is so clearly obvious.
So having a genuine need to be able to actually use tools that you wrote rather than something a company 'licenses' to you so that can modify, and share these tools is being a douche? Odd that you would think that companies that treat their users like untrustworthy hackers are not douches but those users are!
Lots of cross platform goodies in there as well as some interesting data structures. For example, our list data structure is in lines.c, it's extremely small for a small list and scales nicely to 50K items:
This is to answer this question and all the "too late" comments.
Too late? Maybe. But we had a viable business that was pulling in millions/year. The path to giving away our stuff seemed like:
step 1: give it away
step 2: ???
step 3: profit!
And still does. So what changed? Git/Github has all the market share. Trying to compete with that just proved to be too hard. So rather than wait until we were about to turn out the lights, we decided to open source it while we still had money in the bank and see what happens. We've got about 2 years of money and we're trying to build up some additional stuff that we can charge for. We're also open to being doing work for pay to add whatever it is that some company wants to BK, that's more or less what we've been doing for the last 18 years.
Will it work? No idea. We have a couple of years to find out. If nothing pans out, open sourcing it seemed like a better answer than selling it off.
My $0.02 canadian; Build something that kicks Gitlab and Github's ass. What an opportunity. Support both BK and GIT repos. Provide a distributed workflow that enterprises will love. Enterprises are obviously where the remaining dollars are. There are billions of dollars of inefficiencies in that sector. Many of these enterprises do NOT want to host their code on Github and are buying Gitlab. Be better than Gitlab.
As someone who's also read about how Git and Mercurial started (and how Bitkeeper is involved in it), I'm interested in seeing how it will play out. I hope it does work out for you and your team. Thanks for getting it out there.
I'm also interested how open-sourcing BK will improve the other systems, too.
Thanks for providing this level of detail; it's interesting to see the considerations that went into your decision.
How / why did you decide to use the Apache license rather than the GPL?
(It seems like a viral license might protect you a little bit, if you want to prevent your competitors from forking and improving your code base and then using it to compete against you.)
Here's my dream DVCS: easily self-hostable like Fossil, but with good wiki and ticketing system. (Fossil's wiki and ticketing system are awful, but what really sunk it for me was its unexpected behavior for basic commands like "fossil rm".)
I'm just not super-fond of relying on Bitbucket, reliable though they've been, for hosting my stuff.
But a package I could toss on my own VPS? I'd toss some money at that. Wouldn't even need it to be open-source, but I'm no zealot.
i think that bitkeeper could fill in the enterprise niche (like perforce or clearcase) - very big organisations with a lot of developers don't like it that every developer can check out the whole source tree. They usually like to have access control by department/group. Also stuff like 'read only access' or 'right to commit' can be added for greater bureaucratic bliss.
Well clearly in retrospect, step two should have been renaming it "Dawson's Creek Trapper Keeper Ultra Keeper Futura S 2000" [1], adding incredibly advanced computerized features including a television, a music player with voice recognition, OnStar and the ability to automatically hybrid itself to any electronic peripheral device, absorbing the secret military computer at Cheyenne Mountain, and taking over the world.
> Spending a lot of time dealing with manual and bad auto-merges? BitKeeper merges better than most other tools, and you will quickly develop confidence in the quality of the merges, meaning no more reviewing auto-merged code.
Do you have examples of merge-scenarios that are a Conflict for git but resolve for BK?
> BitKeeper’s raw speed for large projects is simply much faster than competing solutions for most common commercial configurations and operations… especially ones that include remote teams, large binary assets, and NFS file systems.
Is there a rule of thumb for what size of repos benefits from BK? (And I suppose size could either be the size of a current commit or the total size of the repo.)
Are there any companies like github or bitbucket that support BitKeeper repos?
I think you guys undersell BAM. That was such a clutch feature where i used BK. It's sad seeing git large file handling just show up, I garuntee it has a long way to go to get parity with BAM.
Amongst all the "too late I loves me some git" type comments, i figure I'd say thankyou and good luck with continued revenue.
I haven't read much about bk so far, so forgive my lazy web question: does/can bk operate over standard ssh as git/hg/svn can, or does it require a dedicated listening server to connect to?
Edit: answering my own question, yes it does support ssh as a transport
I've been using BK/BAM for my photos, it's got 56GB of data in there and works great. I cheat because I added a way to check things out that uses hardlinks instead copies and I can check out the whole tree in 6 seconds. Doing the copy takes a lot longer: 9+ minutes. Hardlinks rock.
On the commercial site there is a link to some BAM paper, take a look at that and maybe ask in the forum or irc if this gets lost.
I half-expected 'very late' comments before I read the comments. I wasn't disappointed.
For those who commented that way, please reconsider this winner takes all approach to your outlook of the world. The world is better because of choice and it's in everybody's best interest to have more distributed version systems.
The argument diversity is good is not so simple true, there are tones of benefits to diversity however there is cost to it too: fragmented finding talent, support, time to fix bugs, more eyes on the project, developer headaches in supporting competing standards so on...
Probably the single biggest reason, aside from it's easier to use than git's CLI, is that it has sub-modules that work exactly like files do in a repository. No extra options, just clone/pull/push/commit/etc. Full on distributed workflow.
BitKeeper itself is a collection of repositories. Download an install image, install, and clone it:
$ bk clone http://bkbits.net/u/bk/bugfix
$ cd bugfix
$ bk here
PRODUCT
default
$ bk comps -m
./src/gui/tcltk/bwidget
./src/gui/tcltk/tcl
./src/gui/tcltk/tk
./src/gui/tcltk/tkcon
./src/gui/tcltk/tktable
./src/gui/tcltk/tktreectrl
./src/win32/dll/scc
./src/win32/dll/shellx
./src/win32/dll/shellx/src
./src/win32/msys
./src/win32/vss2bk
which shows that what we clone by default doesn't include all that other crud (we cache the build result from that and populate it as needed to do builds).
Play with it, it's very different from Git, the subrepo binding is just like file bindings. Everything works together and obeys the same timeline.
BitMover still holds all the copyright, and have all the developers. They obviously wanted to keep BitKeeper proprietary, and are only doing it now when facing irrelevance in the marketplace. If BitKeeper becomes popular again, who’s to say they won't take development proprietary again? Sure, the community could fork the latest free version, but there isn’t a free development community for BitKeeper – they’re all internal to BitMover.
$ bk clone bk://bkbits.net/bkdemo/bk_demo
$ cd bkdemo
# edit files using your favorite editor
$ bk -Ux new
$ bk commit -y"Comments"
$ bk push
As a user whose first CVS was git, I am quite confused by this "quick demo", I have no idea what "-uX" means, no idea what "new" means, no idea what "-y" means and why it is immediately followed by quotation marks instead of being separated by a single space. If bk wants to get new users onboard, it needs a better quick demo that makes sense to new users.
Too late to dominate, but maybe not too late to cut itself a niche. It seems to have some advantages over the competition, and appears to be a reasonable contribution to the table. Besides, competition is always good.
At the very least, Bryan Cantrill will be happy :-D.
Yes. Binaries are handled by one or more servers, we call them BAM servers. The servers hold the data and your repo holds the meta data, binaries are fetched on demand.
You can have a cloud of servers so the binaries are "close" (think China, India, US).
It is unclear to me if the BAM server is part of this opensourcing or not. The page talks about a 90-day trial.
Also, it is common in other (usually non-D)VCS workflows to lock binary files while working on them, since concurrent changes can't be merged the way text files allow. Does BK support anything like this?
Great news! Better late than never! I hope they (or a client of theirs) create a BK backed service soon. I for one, think we need more than just github and altassian in the market if only to ensure the businesses don't take their users for granted (hint: sourceforge)
Huh. Thanks for doing this. As a MySQL employee in the early days I used BitKeeper and fell in love with it and kept using it as long as I could. I mainly use Git these days, but frequently miss BitKeeper -- BK felt a lot more natural to me than Git ever has.
What's not clear from your replies is whether it tracks renames like mercurial, by having users run a manual command to ensure the VCS know about the rename. Except if bk has a file system monitor, I'll assume that's what it does. Unfortunately, data on a few Mercurial repositories I looked at (Mozilla's and Mercurial's) shows that people don't mark all file renames.
How does it detect renamed files in the filesystem?
Thank you for open sourcing by the way, I can definitely see how some features (binary file handling, submodule handling) could be useful for large-scale projects like games.
Looking at the bk import man page, it looks like it cannot import from any modern VCS. I see only RCS, SCCS, CVS, and MKS as options. This is unfortunate, as I have a mercurial tree I'd like to import.
We have a git importer but it's not part of the "official" repo as there are still a few corner cases. I am afraid we don't have a mercurial importer though...
"The ability to seamlessly share only a subset of your source tree "
I've spent a good 10 mins trying to find anything specific in the documentation about this but come up empty. Is this just by virtue of using submodules, ssh and filesystem permissions or is there something more that I'm yet to find? The lack of fine grain security on modern VCS systems is one of the reasons our monolithic repository is still using CVS.
On a related note, the getting started documentation should be more prominent on the Web page.
It's a read only mirror, the read/write mirror is on bkbits.net. But we'll maintain the mirror (or you can, bk has fast-export which creates a perfect mirror in git).
Would you describe the exporting process to git, pretty painless? If so, I'll look at adding bitkeeper support for my git analytics/search tool. I've uploaded some pictures that shows what the git repos that you have hosted on GitHub looks like at:
Since the export process adds "bk: <changeset>" to the commit comment, it'll be easy to tell that it was created via your fast-export tool, which means my tool can easily point you back to the bitkeeper web interface.
The biggest feature for me is the efficient handling of large binary files, because it means I could finally have a completely self-contained repository (clone and everything is in one place, plus free replication), but without the performance penalties which for example Mercurial incurs with binary files:
"[...] Linus moved to it and most of the developers followed. They stayed in it for three more years before moving to Git because BitKeeper wasn't open source."
Glosses over things, but is essentially accurate. Lots of people were not willing to use a proprietary tool, which prompted some reverse-engineering, which caused BK to withdraw their offer.
If all free software activists had accepted the compromise of using the free-as-in-beer BK, git would never have been created.
I think the same points made in Larry's 1993 paper could be made about various Linux distributions:
Why a gazillion package managers?
Why not a common filesystem layout?
Why not a standard desktop?
IMO, Linus should enforce his Linux trademark by forcing every distribution to follow a set of standards. If they don't, they can't call it "Linux". If he got them in a room and said "This is the way it's going to be, or else", they'd do it.
The people that you are looking for are the systemd and FreeDesktop people. The former have a manifesto addressing this:
> The emphasis of systemd to provide a platform instead of just a component allows for closer integration, and cleaner APIs. Sooner or later this will trickle up to the applications. Already, there are accepted XDG specifications [...] that are not supported on the other init systems.
> systemd is also a big opportunity for Linux standardization. Since it standardizes many interfaces of the system that previously have been differing on every distribution, on every implementation, adopting it helps to work against the balkanization of the Linux interfaces.
Interesting — FreeBSD 7 and 8 binaries available for download. Neither of those is a current supported release. It's like offering RHEL 3 or 4 binaries.
We found that by maintaining a build cluster with many old releases we tend to keep compatibility issues out of the code. Also, FreeBSD is very good about backwards compatibility so current releases will run these binaries just fine.
However we will update the build targets as needed by users.
Is that geared towards comparing with Git/Github? Is there a more focused comparison with those. i.e. both comparing to git itself and to GaaS (Git as a Service).
This is very cool... but also, kind of a bit late. The market already adopted git and the momentum is there. Unless there is a trivial way to switch back and forth from git or there is something that is orders of magnitude better, this is a decade too late.
You and a lot of other people say that. Sure, if we want to take over from Git, it's late. But Git has left us with an opening, the only way Git works for the masses is Github, Git itself is too complicated and people "lose" their data (they don't but Git makes it appear like they did).
I think people will play with BK and find out that it can work for everyone without something like Github (we still need it but it's a nice to have, not a requirement).
We'll see. When I was proposing BK the intertubes said it would never work. I'm a little skeptical of the nay sayers.
The "too late" comments are depressing since they totally lack insight (who hasn't noticed the dominations of gits?). Yet also here in HN specifically you would expect people to cheer for pivoting into something new. Thanks for open sourcing BK.
Nah, it's easy. Before every git command, I just tar up the source. When git complains, I can untar to get things working again. To work with others, I fetch a new tree and then use "diff" and "patch" to merge my changes into the new tree.
(seriously, as an experienced professional developer, I actually do this much of the time)
The grand irony is that Larry was one of the earliest advocates of open sourcing the operating system at Sun[1] -- and believed that by the time Sun finally collectively figured it out and made it happen (in 2005), it was a decade or more too late.[2] So on the one hand, you can view the story of BitKeeper with respect to open source as almost Greek in its tragic scope: every reason that Larry outlined for "sourceware"[3] for Sun applied just as much to BK as it did to SunOS -- with even the same technologist (Torvalds) leading the open source alternative! And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005, but I also think this represents a forced dichotomy of "winners" and "losers." To the contrary, I would like to believe that the ongoing innovation in the illumos communities (SmartOS, OmniOS, etc.) proves that it's never too late to open source software -- that open source communities (like cities) can be small yet vibrant, serving a critical role to their constituencies. In an alternate universe, might we be running BK on SunOS instead of git on Linux? Sure -- but being able to run an open source BK on an open source illumos is also pretty great; the future of two innovative systems has been assured, even if it took a little longer than everyone might like.
So congratulations to Larry and crew -- and damn, were you ever right in 1993! ;)
[1] Seriously, read this: http://www.landley.net/history/mirror/unix/srcos.html
[2] The citation here is, in that greatest of all academic euphemisms, "Personal communication."
[3] "Sourceware" because [1] predates the term "open source"
Yeah this irony is not lost on me. But in both cases, the companies acted in self interest. Neither had the guts to walk away from their existing revenue stream. It's hard to say what would have happened.
It's been an interesting ride and if nothing else, BK was the inspiration for Git and Hg, that's a contribution to the field. And maybe, just maybe, people will look at the SCCS weave and realize that Tichy pulled the wool over our eyes. SCCS is profoundly better.
Thank you for contributing to the development and evangalizing of DVCS, directly (BK) and indirectly (the ideas and inspiration for git, hg).
25 replies →
> the companies acted in self interest
Sun, could, at least, make a profit building workstations and servers and licensing chips.
It's actually very sad they don't build those SPARC desktops anymore.
16 replies →
> realize that Tichy pulled the wool over our eyes.
ref: https://en.wikipedia.org/wiki/Revision_Control_System, https://en.wikipedia.org/wiki/Walter_F._Tichy
1 reply →
> in both cases, the companies acted in self interest. Neither had the guts to walk away from their existing revenue stream.
Why does bitmover have the guts now?
1 reply →
I though SCCS had the same problems as RCS. What did it do differently?
35 replies →
I've read that "sourceware" article before, in the distant past when it was still a roughly accurate picture of the market (maybe 1995 or 1996). It's weird to read it again now, in a world that is so remarkably changed. Linux, the scrappy little upstart with a million or so users at the time of the paper, is now the most popular OS (or at least kernel) on the planet, powering billions of phones and servers. NT was viewed as the unfortunate but inevitable future of server operating systems.
I remember looking at IT jobs back then, and seeing a business world covered in Windows NT machines; I even got my MCSE (alongside some UNIX certifications that I was more excited about), because of it. Looking at jobs now, the difference is remarkable, to say the least. Nearly every core technology a system administrator needs to know is Open Source and almost certainly running on Linux.
And, the funny thing is that the general prescription (make a great Open Source UNIX) is exactly what it took to save UNIX. It just didn't involve any of the big UNIX vendors in a significant way (the ones spending a gazillion dollars on UNIX development at the time). Linux got better faster than Sun got smarter, and ate everybody's lunch, including Microsoft. Innovator's Dilemma strikes again.
Apple is an interesting blip on the UNIX history radar, too...though, they're likely to lose to the same market forces in the end, as phones become commodities. I'm a bit concerned that it's going to be Android, however, that wins the mobile world since Android is nowhere near the ideal OS from an Open Source and ethical perspective; but, I guess they got the bits right that Larry was suggesting needed to be right.
Anyway, it was a weird flashback to read that article. Things change, and on a scale that seems slow, until you look back on it, and see it's "only" been a couple of decades. In the grand scheme of things, and compared to the motion of technology prior to the 1900, that's a blink of an eye.
Eh. Linux does have a LOT of problems. Well, so does everything, but it's not like it's "great." More like "good."
But yeah, it was weird that everybody thought NT was going to be the future. And now, MS has opensourced a good deal of infrastructure, is working with Node, has announced an integrated POSIX environment for Windows. And since it's in corporate, it might even be able to fix the fork(2) performance problems.
23 replies →
"I'm a bit concerned that it's going to be Android, however, that wins the mobile world since Android is nowhere near the ideal OS from an Open Source and ethical perspective; but, I guess they got the bits right that Larry was suggesting needed to be right."
They've already won; Apple isn't coming back (did they "go thermonuclear" in the end or did that nonsense die with Mr Magical Thinking?).
Don't confuse Android with Google; you can grab the source and do what you want with it, like Cyanogenmod have, or like millions of hobbyists are doing themselves. It's all available with bog standard open source licenses - no need to worry about ethics.
2 replies →
This is a bit funny because Mercurial is partly named after Larry.
https://groups.google.com/d/msg/mercurial_general/c3_SM3p7S1...
Just for the record this thread is where I learned that. And yup, it sorta fits. For better or worse.
The point about small communities is great. I've never seen it spelled out like that, but it captures what I was thinking perfectly. People get caught up on popularity as the only meaningful metric for success, but it really isn't.
> And you can say to BK and Larry now that it's "too late", just as Larry told Sun in 2005,
In case of software, it is never too late: as you once put it so eloquently, software does not suddenly stop working and does not have an expiration date.
If this software works and works well, then Paul Graham's revolutionary idea of when you choose technology, you have to ignore what other people are doing, and consider only what will work the best applies. (Common sense really, but apparently not to the rest of our industry.)
If this software will work the best, and do exactly what I want and need it to do, I have enough experience to know not to care that everyone else runs something like git just because that is trendy right now. (A lesson appreciated by those who run SmartOS because it is the best available technology for virtualization, cloud, and performance, instead of running Linux and Docker.)
> software [...] does not have an expiration date.
xscreensaver does. (-:
* https://news.ycombinator.com/item?id=11412081
Sun is also no more... and it's not at all clear they would've survived had Solaris been open sourced a decade earlier.
Yes, you're absolutely right that there are a ton of startups built on opensolaris (who have proprietary code they haven't and don't intend to ever give back to the community), and there is smartos/omnios/illumos as well. But none of those projects would have in any way contributed to the health of Sun Microsystems, nor provided the funding to get Solaris to where it is today. ZFS may have never seen the light of day if Solaris were open sourced in 1995.
> ZFS may have never seen the light of day if Solaris were open sourced in 1995.
It depends on how that would have affected Jeff Bonwick. If it kept him from deciding that Sun ought to develop a new filesystem, promising Matthew Ahrens a job writing one out of college and working together with Matt on it, ZFS would never have existed.
2 replies →
I actually posted before I saw your comment. I guess I was right. So are you happy to finally have an open source bring-over-modify-merge VCS whose command set makes sense?
Bit keeper is a great example of what happens when you do not open source your code. I have cited it that way many times.
Except that we've been around for 18 years and made payroll without fail that entire time. Supported a team of 10-15 people every year. That's something, many many companies in the valley, including many that open sourced everything, have not done as well.
You may have done more by open sourcing whatever it is that you have done; if so congrats.
4 replies →
I have no idea why people are down voting this. In an alternative universe, we would all be using bit keeper. The reason we are not is mainly because Larry McVoy ceded the market to git and mercurial because he was afraid of disrupting his existing business. git and mercurial would never have existed had he practiced at Bit Movers what he preached at Sun.
For people who don't know the history -- McVoy offered free bitkeeper licenses to various open source projects, and the Linux kernel switched to it.
After Andrew Tridgell (SAMBA, among other projects) reverse-engineered the bitkeeper protocol [1] in order to create his own client, the license was rescinded for everyone.
As a result, Linus wrote git.
[1] https://lwn.net/Articles/132938/
> As a result, Linus wrote git.
And mpm wrote hg, never forget:
http://lkml.iu.edu/hypermail/linux/kernel/0504.2/0670.html
http://lwn.net/Articles/151624/
It's astonishing to me that Git has won out given how much easier it's been for me to explain Hg to other people than to explain Git. To this day, in our SVN workflow at my company, nontechnical people who have merely seen a Hg diagram on a whiteboard by my desk immediately grasped the idea and the lingo, and ask me questions like "hey, can you branch the code to commit those changes and push them to the testing server? This thing's really cool and we don't mind playing with the alpha version, but we might scrap it all later."
30 replies →
From what I read, he connected to a bit keeper repository via telnet on port 5000, executed the help command and then used that information to write an incomplete client. That does not sound like reverse engineering to me.
It is reverse engineering. It's just easy reverse engineering.
As I remember it, it was a bit of a douche move by Tridgell, driven by a Stallman-like free software ideology.
It wasn't. He gave conclusive reply which established that it's ethical (just telneting and help). Unless you believe samba and everything else is unethical and you club every reverse engineering under one umbrella, your comment is wrong. http://www.theregister.co.uk/2005/04/14/torvalds_attacks_tri...
I don't think it's fair to call people douches because they are committed to their moral principals. Especially so here, where the benefit to humanity over the alternative is so clearly obvious.
2 replies →
> a Stallman-like free software ideology
You say that like it's a bad thing.
As I remember it, he did
telnet bk-server 5000
and typed "help".
https://lwn.net/Articles/132938/
1 reply →
So having a genuine need to be able to actually use tools that you wrote rather than something a company 'licenses' to you so that can modify, and share these tools is being a douche? Odd that you would think that companies that treat their users like untrustworthy hackers are not douches but those users are!
Here's an article about that: http://www.theregister.co.uk/2005/04/14/torvalds_attacks_tri...
Lots of cross platform goodies in there as well as some interesting data structures. For example, our list data structure is in lines.c, it's extremely small for a small list and scales nicely to 50K items:
http://bkbits.net/u/bk/bugfix/src/libc/utils/lines.c?PAGE=an...
1 year ago: https://news.ycombinator.com/item?id=9330482
What changed? Is BitKeeper still an ongoing business with some other model, or is that, as they say... it? I hope not.
This is to answer this question and all the "too late" comments.
Too late? Maybe. But we had a viable business that was pulling in millions/year. The path to giving away our stuff seemed like:
And still does. So what changed? Git/Github has all the market share. Trying to compete with that just proved to be too hard. So rather than wait until we were about to turn out the lights, we decided to open source it while we still had money in the bank and see what happens. We've got about 2 years of money and we're trying to build up some additional stuff that we can charge for. We're also open to being doing work for pay to add whatever it is that some company wants to BK, that's more or less what we've been doing for the last 18 years.
Will it work? No idea. We have a couple of years to find out. If nothing pans out, open sourcing it seemed like a better answer than selling it off.
My $0.02 canadian; Build something that kicks Gitlab and Github's ass. What an opportunity. Support both BK and GIT repos. Provide a distributed workflow that enterprises will love. Enterprises are obviously where the remaining dollars are. There are billions of dollars of inefficiencies in that sector. Many of these enterprises do NOT want to host their code on Github and are buying Gitlab. Be better than Gitlab.
10 replies →
As someone who's also read about how Git and Mercurial started (and how Bitkeeper is involved in it), I'm interested in seeing how it will play out. I hope it does work out for you and your team. Thanks for getting it out there.
I'm also interested how open-sourcing BK will improve the other systems, too.
4 replies →
Thanks for providing this level of detail; it's interesting to see the considerations that went into your decision.
How / why did you decide to use the Apache license rather than the GPL?
(It seems like a viral license might protect you a little bit, if you want to prevent your competitors from forking and improving your code base and then using it to compete against you.)
7 replies →
Here's my dream DVCS: easily self-hostable like Fossil, but with good wiki and ticketing system. (Fossil's wiki and ticketing system are awful, but what really sunk it for me was its unexpected behavior for basic commands like "fossil rm".)
I'm just not super-fond of relying on Bitbucket, reliable though they've been, for hosting my stuff.
But a package I could toss on my own VPS? I'd toss some money at that. Wouldn't even need it to be open-source, but I'm no zealot.
4 replies →
i think that bitkeeper could fill in the enterprise niche (like perforce or clearcase) - very big organisations with a lot of developers don't like it that every developer can check out the whole source tree. They usually like to have access control by department/group. Also stuff like 'read only access' or 'right to commit' can be added for greater bureaucratic bliss.
There is plenty of people doing "lets turn opensource and blame the community if does not work" arround.
It's a good excuse to blame that opensource broke your business, and that opensource could not save you from dying...
A South Park reference, ehe?
Well clearly in retrospect, step two should have been renaming it "Dawson's Creek Trapper Keeper Ultra Keeper Futura S 2000" [1], adding incredibly advanced computerized features including a television, a music player with voice recognition, OnStar and the ability to automatically hybrid itself to any electronic peripheral device, absorbing the secret military computer at Cheyenne Mountain, and taking over the world.
[1] https://en.wikipedia.org/wiki/Trapper_Keeper_(South_Park)
I have some questions about Why.html: https://www.bitkeeper.org/why.html
> Spending a lot of time dealing with manual and bad auto-merges? BitKeeper merges better than most other tools, and you will quickly develop confidence in the quality of the merges, meaning no more reviewing auto-merged code.
Do you have examples of merge-scenarios that are a Conflict for git but resolve for BK?
> BitKeeper’s raw speed for large projects is simply much faster than competing solutions for most common commercial configurations and operations… especially ones that include remote teams, large binary assets, and NFS file systems.
Is there a rule of thumb for what size of repos benefits from BK? (And I suppose size could either be the size of a current commit or the total size of the repo.)
Are there any companies like github or bitbucket that support BitKeeper repos?
Wayne pointed to some stuff over on the reddit thread.
As for size it's csets * files, as that gets big, Git slows down faster than linear, we're pretty linear.
I think you guys undersell BAM. That was such a clutch feature where i used BK. It's sad seeing git large file handling just show up, I garuntee it has a long way to go to get parity with BAM.
Amongst all the "too late I loves me some git" type comments, i figure I'd say thankyou and good luck with continued revenue.
I haven't read much about bk so far, so forgive my lazy web question: does/can bk operate over standard ssh as git/hg/svn can, or does it require a dedicated listening server to connect to?
Edit: answering my own question, yes it does support ssh as a transport
How does BitKeeper scale to large projects? (Like, say, gigabytes of binaries.) This is a weak area of Git.
---
From the "Why" page:
BitKeeper’s Binary Asset Manager (BAM) preserves resources and keeps access fast by providing local storage as needed.
BAM is great for any organization that handles:
* Videos
* Photos
* Artwork
* Office files
* CAD files
* Any large binary files
I've been using BK/BAM for my photos, it's got 56GB of data in there and works great. I cheat because I added a way to check things out that uses hardlinks instead copies and I can check out the whole tree in 6 seconds. Doing the copy takes a lot longer: 9+ minutes. Hardlinks rock.
On the commercial site there is a link to some BAM paper, take a look at that and maybe ask in the forum or irc if this gets lost.
Wonder if BitKeeper might be a viable alternative for Git LFS (https://git-lfs.github.com) then.
I half-expected 'very late' comments before I read the comments. I wasn't disappointed.
For those who commented that way, please reconsider this winner takes all approach to your outlook of the world. The world is better because of choice and it's in everybody's best interest to have more distributed version systems.
late does not mean it is useless.
The argument diversity is good is not so simple true, there are tones of benefits to diversity however there is cost to it too: fragmented finding talent, support, time to fix bugs, more eyes on the project, developer headaches in supporting competing standards so on...
Why would I want to use this over git or mercurial?
You wouldn't, unless you have very specific niche needs. They're pretty upfront about it:
>Why use BitKeeper when there are lots of great alternatives?
>For many projects, the answer is: you shouldn’t.
https://www.bitkeeper.org/why.html
Probably the single biggest reason, aside from it's easier to use than git's CLI, is that it has sub-modules that work exactly like files do in a repository. No extra options, just clone/pull/push/commit/etc. Full on distributed workflow.
BitKeeper itself is a collection of repositories. Download an install image, install, and clone it:
which shows that what we clone by default doesn't include all that other crud (we cache the build result from that and populate it as needed to do builds).
Play with it, it's very different from Git, the subrepo binding is just like file bindings. Everything works together and obeys the same timeline.
17 replies →
BitMover still holds all the copyright, and have all the developers. They obviously wanted to keep BitKeeper proprietary, and are only doing it now when facing irrelevance in the marketplace. If BitKeeper becomes popular again, who’s to say they won't take development proprietary again? Sure, the community could fork the latest free version, but there isn’t a free development community for BitKeeper – they’re all internal to BitMover.
As a user whose first CVS was git, I am quite confused by this "quick demo", I have no idea what "-uX" means, no idea what "new" means, no idea what "-y" means and why it is immediately followed by quotation marks instead of being separated by a single space. If bk wants to get new users onboard, it needs a better quick demo that makes sense to new users.
According to http://www.bitkeeper.com/testdrive:
* The -U option to bk tells it to operate on "user files". That is files that are not part of the BitKeeper metadata
* The modifier x corresponds to "extras", files which Bitkeeper doesn't know about (changed files is c)
* `new` adds files to the repository
* [on commit] the -y option is for changeset comments (~commit messages)
So `bk -Ux new` is `git add <untracked files>`[0] and `bk commit -y"thing"` is `git commit -m "thing"`
[0] aka `git add $(git ls-files -o --exclude-standard)` or `git add -i<RET> <a> <*> <q>`
Too late to dominate, but maybe not too late to cut itself a niche. It seems to have some advantages over the competition, and appears to be a reasonable contribution to the table. Besides, competition is always good.
At the very least, Bryan Cantrill will be happy :-D.
As you can see. :-)
https://news.ycombinator.com/item?id=11668678
I'm wondering: how does it handle large binary files? Any better than git or hg without extensions?
Yes. Binaries are handled by one or more servers, we call them BAM servers. The servers hold the data and your repo holds the meta data, binaries are fetched on demand.
You can have a cloud of servers so the binaries are "close" (think China, India, US).
Two questions:
It is unclear to me if the BAM server is part of this opensourcing or not. The page talks about a 90-day trial.
Also, it is common in other (usually non-D)VCS workflows to lock binary files while working on them, since concurrent changes can't be merged the way text files allow. Does BK support anything like this?
3 replies →
Great news! Better late than never! I hope they (or a client of theirs) create a BK backed service soon. I for one, think we need more than just github and altassian in the market if only to ensure the businesses don't take their users for granted (hint: sourceforge)
There are tons of alternatives to github/atlassian/sourceforge:
https://en.wikipedia.org/wiki/Comparison_of_open_source_soft...
Huh. Thanks for doing this. As a MySQL employee in the early days I used BitKeeper and fell in love with it and kept using it as long as I could. I mainly use Git these days, but frequently miss BitKeeper -- BK felt a lot more natural to me than Git ever has.
Hey Jeremy, long time no talk. We have the MySQL 6.x tree in BK, we can put up on bkbits if you like.
Something I'm wondering and the man page doesn't clear, does it track files across renames or does it only track content like git?
It tracks renames, it's not like git. Every file has an internal identifier, that's the actual file id, the name is a versioned attribute of the file.
What's not clear from your replies is whether it tracks renames like mercurial, by having users run a manual command to ensure the VCS know about the rename. Except if bk has a file system monitor, I'll assume that's what it does. Unfortunately, data on a few Mercurial repositories I looked at (Mozilla's and Mercurial's) shows that people don't mark all file renames.
1 reply →
How does it detect renamed files in the filesystem?
Thank you for open sourcing by the way, I can definitely see how some features (binary file handling, submodule handling) could be useful for large-scale projects like games.
6 replies →
Yes, files are tracked as first class objects as they are renamed.
Can it import from git or SVN or mercurial?
Looking at the bk import man page, it looks like it cannot import from any modern VCS. I see only RCS, SCCS, CVS, and MKS as options. This is unfortunate, as I have a mercurial tree I'd like to import.
We have a git importer but it's not part of the "official" repo as there are still a few corner cases. I am afraid we don't have a mercurial importer though...
You probably should publish the git importer -- it makes it really hard to try bitkeeper if you have to play with a pretend tree.
3 replies →
Well, that took a long time... I wonder what changed in the eleven years that Git and Mercurial were deployed to replace bitkeeper.
I've spent a good 10 mins trying to find anything specific in the documentation about this but come up empty. Is this just by virtue of using submodules, ssh and filesystem permissions or is there something more that I'm yet to find? The lack of fine grain security on modern VCS systems is one of the reasons our monolithic repository is still using CVS.
On a related note, the getting started documentation should be more prominent on the Web page.
There is an official mirror on GitHub:
https://github.com/bitkeeper-scm/bitkeeper
It's a read only mirror, the read/write mirror is on bkbits.net. But we'll maintain the mirror (or you can, bk has fast-export which creates a perfect mirror in git).
Would you describe the exporting process to git, pretty painless? If so, I'll look at adding bitkeeper support for my git analytics/search tool. I've uploaded some pictures that shows what the git repos that you have hosted on GitHub looks like at:
http://imgur.com/a/nVvov
Since the export process adds "bk: <changeset>" to the commit comment, it'll be easy to tell that it was created via your fast-export tool, which means my tool can easily point you back to the bitkeeper web interface.
Is there a way to do the reverse, and convert a Git repository into a BitKeeper repository? I found the fast-export manual page:
https://www.bitkeeper.org/man/fast-export.html
But it doesn't look like there's any fast-import. Do you have some recommended way if I wanted to convert an existing repository to try BitKeeper out?
By "fast-export", you mean Packard's "fast-import" format, editable with ESR's RepoSurgeon?
The biggest feature for me is the efficient handling of large binary files, because it means I could finally have a completely self-contained repository (clone and everything is in one place, plus free replication), but without the performance penalties which for example Mercurial incurs with binary files:
https://www.bitkeeper.org/why.html
I have to try it out just for that!
This predates git, in fact if it was open sourced from the start git may never have existed, sigh, how ironic.
If bitkeeper was open sourced it could be a powerhouse nowadays, open source and commercially. Now it is too late and honestly irrelevant.
"[...] Linus moved to it and most of the developers followed. They stayed in it for three more years before moving to Git because BitKeeper wasn't open source."
Um, the "because" part is not quite right.
Glosses over things, but is essentially accurate. Lots of people were not willing to use a proprietary tool, which prompted some reverse-engineering, which caused BK to withdraw their offer.
If all free software activists had accepted the compromise of using the free-as-in-beer BK, git would never have been created.
That's not:
They stayed in it for three more years [...] because BitKeeper wasn't open source.
but
They stayed in it for three more years before [moving to Git because BitKeeper wasn't open source].
I think the same points made in Larry's 1993 paper could be made about various Linux distributions:
IMO, Linus should enforce his Linux trademark by forcing every distribution to follow a set of standards. If they don't, they can't call it "Linux". If he got them in a room and said "This is the way it's going to be, or else", they'd do it.
The people that you are looking for are the systemd and FreeDesktop people. The former have a manifesto addressing this:
> The emphasis of systemd to provide a platform instead of just a component allows for closer integration, and cleaner APIs. Sooner or later this will trickle up to the applications. Already, there are accepted XDG specifications [...] that are not supported on the other init systems.
> systemd is also a big opportunity for Linux standardization. Since it standardizes many interfaces of the system that previously have been differing on every distribution, on every implementation, adopting it helps to work against the balkanization of the Linux interfaces.
-- https://ti.to/systemdconf/systemdconf-2016
Some history from Linus himself https://www.youtube.com/watch?v=4XpnKHJAok8
Interesting — FreeBSD 7 and 8 binaries available for download. Neither of those is a current supported release. It's like offering RHEL 3 or 4 binaries.
We found that by maintaining a build cluster with many old releases we tend to keep compatibility issues out of the code. Also, FreeBSD is very good about backwards compatibility so current releases will run these binaries just fine.
However we will update the build targets as needed by users.
+1 For FreeBSD support +1 For open sourcing BK
I hope it works in your best interest. And I wish you all at BK the absolute best and thank you for all your incredibly hard work over the years.
BTW, sorry to say we don't have a RHEL 3 release for you but in the 'complete list' are you can find stuff for RHEL 4. ;-)
Really they are Debian 4, 5, 6, 7, & 8 but they match up with Redhat pretty well.
RHEL 4 is still supported. Extended support is good for another year.
I see this as "features" https://www.bitkeeper.org/why.html
See large repo support, security and others.
Is that geared towards comparing with Git/Github? Is there a more focused comparison with those. i.e. both comparing to git itself and to GaaS (Git as a Service).
The nested repository feature sounds amazing. Dealing with both git submodules and git subtrees has been a huge pain for me.
I'm looking forward to trying this out over the weekend. Is there some kind of util/script to import history from git?
Working on getting a crappy one out there, we have simple and pretty crappy and complex / fragile but less crappy.
Does this come with any sort of web interface?
Yes, there is hosting at bkbits.net and if you drill down to
http://bkbits.net/u/bk/bugfix/
that's bk/web which is included in the release.
Great to see this finally happen... However, for 'us' Git remains a keeper.
This is very cool... but also, kind of a bit late. The market already adopted git and the momentum is there. Unless there is a trivial way to switch back and forth from git or there is something that is orders of magnitude better, this is a decade too late.
'A bit late' might be understatement of the day :)
Too late?
Too late :)
You and a lot of other people say that. Sure, if we want to take over from Git, it's late. But Git has left us with an opening, the only way Git works for the masses is Github, Git itself is too complicated and people "lose" their data (they don't but Git makes it appear like they did).
I think people will play with BK and find out that it can work for everyone without something like Github (we still need it but it's a nice to have, not a requirement).
We'll see. When I was proposing BK the intertubes said it would never work. I'm a little skeptical of the nay sayers.
The "too late" comments are depressing since they totally lack insight (who hasn't noticed the dominations of gits?). Yet also here in HN specifically you would expect people to cheer for pivoting into something new. Thanks for open sourcing BK.
Nah, it's easy. Before every git command, I just tar up the source. When git complains, I can untar to get things working again. To work with others, I fetch a new tree and then use "diff" and "patch" to merge my changes into the new tree.
(seriously, as an experienced professional developer, I actually do this much of the time)
2 replies →
It's too late. There is no reason to use non-git DVCS in 2016.