It's nice to see that internal developers feel the same way about XNA that external developers (who used to build XNA games, or still build XNA games) do.
From the outside I always assumed the constant flood of new, half-baked features instead of fixes and improvements to old ones was caused by interns and junior devs looking for glory - sad to hear that's actually partly true. I always considered frameworks like WPF (or Flex, for that matter) 'intern code' - not that interns necessarily wrote them, but they reek of not-experienced-enough engineers trying to solve problems by writing a bunch of new code, instead of fixing existing code.
It really is too bad, though. There are parts of the NT kernel (and even the Win32 API) that I consider a joy to use - I love IOCP, despite its warts, and APIs like MsgWaitForMultipleObjects are great tools for building higher-level primitives.
Plus, say what you want about GDI (there's a lot wrong with it at this point), but it's still a surprisingly efficient and flexible way to do 2D rendering, despite the fact that parts of it date back to before Windows 3.1. Some really smart people did some really good API design over time over at Microsoft...
Actually, I think one NT's largest advantages over POSIX systems is process management: yes, the venerable CreateProcess API.
See, in Windows, processes are first class kernel objects. You have handles (read: file descriptors) that refer to them. Processes have POSIX-style PIDs too, but you don't use a PID to manipulate a process the way you would with kill(2): you use a PID to open a handle to a process, then you manipulate the process using the handle.
This approach, at a stroke, solves all the wait, wait3, wait4, SIGCHLD, etc. problems that plague Unixish systems to this day. (Oh, and while you have a handle to a process open, its process ID won't be re-used.)
It's as if we live in a better, alternate universe where fork(2) returns a file descriptor.
You can wait on process handles (the handle becomes signaled and the wait completes when the process exits). You can perform this waiting using the same functions you use to wait on anything else, and you can use WaitForMultipleObjects as a kind of super-select to wait on anything.
If you want to wait on a socket, a process, and a global mutex and wake up when any of these things becomes available, you can do that. The Unix APIs for doing the same thing are a mess. Don't even get me started on SysV IPC.
You can apply memory use, scheduling, UI, and other restrictions to processes in job objects. Most conveniently of all, you can arrange for the OS to kill everything in a job object if the last handle to that job dies --- the closest Linux has is PR_SET_PDEATHSIG, which needs to be set up individually for each child and which doesn't work for setuid children.
(Oh, and you can arrange for job objects to send notifications to IO completion ports.)
Yes, Windows gets a lot wrong, but it gets a lot right.
> WaitForMultipleObjects as a kind of super-select to wait on anything
Except for the 64 handle limit, which makes it largely useless for anything that involves server applications where the number of handles grows with the number of clients. So then you'd spawn "worker" threads, each handling just 64 handles. And that's exactly where your code starts to go sour - you are forced to add fluff the sole purpose of which is to work around API limitations. What good is the super-select if I need cruft to actually use it in the app?
And don't get me started on the clusterfuck of the API that is IOCP.
> Yes, Windows gets a lot wrong, but it gets a lot right.
Oh, no, it doesn't.
Specifically, what Windows does not get is that it's other developers that are using its services and APIs, not just MSSQL team that can peek at the source code and cook up a magic combination of function arguments that does not return undocumented 0x8Fuck0ff. I have coded extensively for both Linux and Windows, including drivers and various kernel components, and while Windows may get things right at the conceptual level, using what they end up implemented as is an inferior and painful experience.
My personal annoyance is the introduction of win32k in NT4. With per-session CSRSS existing since WinFrame and Vista+ having Session 0 Isolation, not to mention much faster processors, and the removal of XPDM and the requirement of the DWM in Win8, it doesn't make as much sense anymore.
Update: And I forgot to mention the font parsing security issues (look up Duqu), and the issues with user mode callbacks too.
> I always considered frameworks like WPF (or Flex, for that matter) 'intern code' - not that interns necessarily wrote them, but they reek of not-experienced-enough engineers trying to solve problems by writing a bunch of new code, instead of fixing existing code.
This is unfair and unfounded accusation. If you look at the time frame, all the major platform were working on their first hardware accelerated UI toolkits and went through the similar teething problems (Cocoa anyone?). WinForms was a dead end, there was no fixing to do. WPF has turned out well enough, and WinRT has evolved into something very efficient (e.g. by using ref counting rather than GC).
> Plus, say what you want about GDI (there's a lot wrong with it at this point), but it's still a surprisingly efficient and flexible way to do 2D rendering
Not anymore. I get it that some love to use antique APIs and computer systems just for the sake of being retro, but when every computer ships these days with a GPU, using GDI is not even close to pragmatic.
The "Inside Windows Kernel" book series are quite interesting to understand on how it all works, and even some of the initial VMS influence in the original kernel design.
> "It's nice to see that internal developers feel the same way about XNA that external developers (who used to build XNA games, or still build XNA games) do. From the outside I always assumed the constant flood of new, half-baked features instead of fixes and improvements to old ones was caused by interns and junior devs looking for glory ..."
Am I understanding you correct; are you implying that you think XNA was created by juniors? Would you say that because of this, it's a good thing XNA is being killed?
Cause personally I think it's been a terrible decision by Microsoft to kill XNA. A lot of indie game developers have relied on XNA and I really feel Microsoft can use the indie support. Sure, big name games might be a priority, but personally I feel most _interesting_ work is being done by indies. Indies tend to be less concerned with proven formulas and seem to see it more of a creative outlet for themselves[1]. I think it's a good thing frameworks like Monogame[2] exist, so developers can still put their existing XNA knowledge to good use - and not just limited to the Windows platform, but e.g. iOS and Android as well.
The Monogame website might not show very impressing examples, but a game like Bastion[3] was ported to other platforms using Monogame, showing very high-quality work can be created with XNA.
What is the feeling about XNA? I haven't followed the area, so I found it unfortunate that it was treated in the original post without explanation. Was XNA a good thing or a bad thing? Why?
It was loved by Indies for XBox 360 development, got so popular, Microsoft decided to make it the way to develop games on the Windows Phone 7.
When Windows Phone 8 came out, the native renaissance was full speed inside Microsoft, C++ with DirectX became the official way and XNA was canned while Microsoft left MonoGame and Unity pick up the C# developers still willing to invest into the platform.
The sad thing is that this is not a native vs managed issue, because even C# gets compiled to native code when targeting Windows Phone 8. Just plain product management decision to kill the product, which was being sold as the way to develop Windows Phone games.
> Another reason for the quality gap is that that we've been having trouble keeping talented people. Google and other large Seattle-area companies keep poaching our best, most experienced developers, and we hire youths straight from college to replace them.
I will say all of the ex-Microsoft folks I've encountered at Google Seattle have been fantastic.
On a related note, it's stupidly easy to get code accepted by another team at Google.
Too bad you guys don't seem to give a second glance at someone without any formal education, despite working on systems for government, training, banking security and airline industries.
I have zero formal education past A levels, but I've had two in-person interviews at Google and had an offer, and I have their charming recruiters chasing me to make sure I wouldn't be happier working for Google every 6 months or so. :)
It does take a ferociously long time to get through the initial screening if you just send them a CV, and I'm sure a lot of people fall through the cracks - my suggestion would be to find a Googler or Xoogler and ask them to recommend you, I understand you get a lot more traction that way. Chin up.
"Too bad you guys don't seem to give a second glance at someone without any formal education, despite working on systems for government, training, banking security and airline industries."
I find it doubtful that your lack of education alone is what's keeping you from getting a callback. Do you have a portfolio linked from your resume? Have you got friends that work for these companies that are willing to go over your resume with you?
> On a related note, it's stupidly easy to get code accepted by another team at Google.
Unless that other team is Android. Though then you could submit to AOSP directly (assuming the issue you are addressing wasn't fixed internally 4 months ago, but how would you know?).
I look forward to further entanglement with a hiring process much more toxic than Microsoft's. Plus the joy of being hired for a mystery job where all my Big O knowledge goes to writing scripts for placing free porn ads.
I actually enjoy working at Microsoft --- I'm in Phone, working on telemetry and various other things --- and I've met a ton of very smart people people. I've also made the cross-team (and cross-org) contributions that the OP says are nearly impossible (just today, even). While the OP makes a few good points (some teams are kinda reluctant to take patches), I think he's grossly exaggerating the problems, and the level of vitriol really isn't called for either.
He's also slightly off-base on some of the technical criticisms: there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent. Besides, change _does_ happen: Arun Kishan (who is smarter than I'll ever be) broke the dispatcher lock a while ago (see http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewe...) when nobody thought it could be done.
By the way: there's actually quite a lot of information available on Windows Internals. In fact, there's a whole book, Windows Internals (http://technet.microsoft.com/en-us/sysinternals/bb963901.asp...) on exactly how the NT kernel and related system components work. The book is highly recommended reading and demystifies a lot of the odder parts of our API surface. We use it internally all the time, and reading it, you'll repeatedly say to yourself, "Ah, so that's why Foo does Bar and not Qux! Yeah, that's a good reason. I didn't think of it that way."
"there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent."
One of the issues the author seemed to be alluding to is that the loss of experienced developers makes it really hard to keep track of "line 123 of foo_dispatch.c does this because of vital business reason xyz or compatibility reason uvw" and "line 123 of foo_dispatch.c was written while I was hungover and trying to make a deadline--it looks clever, but feel free to junk it.".
This issue is compounded when you are hiring safe, and you have a culture where making gross fuckups while trying to make progress is discouraged. It is neither good nor bad--after all, I too enjoy using stable software--but there is a price to be paid if devs don't feel comfortable making breaking changes.
There are always good reasons for doing everything. But this is how you end up with a "death by a thousand cuts" scenario. Sometimes, if you want to be great, you have to be bold. Windows NT was bold back in 1995 when I first played with 3.51. It was the best OS out there by far.
Now, once you set up a system where people play it safe more than they innovate, you'll get a situation where all of the best programmers will want to leave because it's boring, and you're left with mediocre programmers who aren't good enough to be bold and innovative. This is exactly what the OP describes.
Look at recent Microsoft releases: we don't fix old features, but accrete new ones. New features help much more at review time than improvements to old ones.
(That's literally the explanation for PowerShell. Many of us wanted to improve cmd.exe, but couldn't.)
Ahh, I was wondering about that. So, I guess I'll just keep using cygwin.
On another note, I recently asked a friend who works at Microsoft how work is going. His reply: "Well, it's calibration time, so lots of sucking up to the boss." Must be hard to get much actual work done when you're worried about that all the time.
I really can't agree with that. To me, Powershell feels like it was designed by geniuses and implemented by interns.
Absolutely basic stuff, like "find the path of this file" (%~dp0 in batch) is a 5 line copy-paste. Why? It's lovely that I can pipe .NET objects places, but every time i want it to actually do something, I end up in copy-paste hell.
I don't understand why they felt the need to defy standards with powershell. It's not similar to anything else i'm familiar with. All they had to do was copy bash and port some of the GNU tools.
As an ex-MS employee I always feel obligated to point out that MS has well over 50,000 employees spread out across countless teams and divisions. One employee's experience is never indicative of the company as a whole. I never went through "calibration time" while at MS or anything even close to it.
I do wonder, however, whether 'boo! MS's rating system is suboptimal, to say the least!' is a very useful comment here (which might by why we haven't seen Michael write here yet)
Yeah, and that was about Google, which has a much better reputation than Microsoft in this area. I wonder how he would like working at MS. I doubt he'd last a week there!
I'd agree. I know the OP is trying to spread knowledge and this is a great read, but I think this was slightly rude, not to mention questionable fair use.
There's no "fair use" argument at all. Copying an entire work, then distributing it with no transformative change or criticism, is inexcusable infringement should the author want to enforce his rights.
If your first priority is your kids, your first priority is not your work. That's fine, it's a choice, but there is a frequent claim that the massive incremental time demand of kids makes one so much more time efficient at work that it more-than-compensates. Insofar as one is the same human being, with the same energy reservoirs and time management skills as before your child was born, this is unlikely.
I understand the reason this fiction is maintained: people with kids need the job even more than people without, and have an interest in denouncing people who claim kids make you less productive. There's also the second order effect in which "with kids" is correlated with "older".
The net of it though is that devs with kids tend to assign work a lower priority, to take fewer risks, to need more money, and to be older and hence less familiar with new technologies (and too busy to learn in their free time).
Society doesn't have a good answer for this situation yet. In times past, technology didn't move so fast that experience was mostly obsolete (and hence useless) in a decade's time. A 40 year old farmer with kids in 1713 would probably have much to teach a young whippersnapper. The same isn't true for a 40 year old programmer with kids in 2013.
LOL at technology moving fast and experience becoming obsolete in "a decade's time." I'm not even that old--I started programming in the late 1990's--but I've been programming more than a decade, and as far as I can tell all that has happened to mainstream programming technology in that time span is bike-shedding. There has been basically nothing new in mainstream programming since about the 1980's ("Oh, NewtonScript looks like C now instead of Pascal, and we're calling it Javascript? That's nice.") In many ways it's gone backwards (the sorry state of Javascript IDE's relative to Visual Studio 6.x not to mention Smalltalk IDE's a decade older than even that). "Oh, you can use threads in web apps now? Like you could in Win32 apps since 1995?"
To a first approximation, all this new "web technology" is just a way of doing what you could've done on an internet-connected NeXT machine 20 years ago, only more brain-damaged and frustrating to work with.
The only place technology has advanced in that time period is in domain-specific areas (which of course requires experts, not young whippersnappers), and infrastructure (the world looks a lot different for computers with pervasive 4G--the software is just an adaptation of proven concepts to uses enabled by innovative hardware).
Programming is applied mathematics, and that hasn't changed substantially since people were developing software on punch cards.
When will we get it that in real life, people don't have "first priorities" just as they don't have "arch enemies"? :) Really, any sane not-overworked not-burned-out team will benefit from older more experienced devs in it. If you're overworking the shit out of everybody, of course that the older-with-kids guys will underperform. But if you are overworking people like hell, it probably just means that you are doing things very inefficiently and compensating by "sheer work power" and shooting yourself in the foot by making the experience holders work at 40% of their top performance because of the environment (ok, I agree, this might be a good trade-off for some businesses). Chill out a bit, do things the right way and at the "natural" pace and you don't need to care about this "problem" or use the nine-to-five-with-kids stereotype. As one wise man told me once: you can't "move fast and break things" kiddo, because all that time spent fixing the things you break actually means that you're not moving as fast as you can or fast enough.
> experience was mostly obsolete (and hence useless) in a decade's time
I'm 28, which means I'm young enough that even people like you believe I can learn new tricks. As far as I can tell, my brain is still fully functional. That said, most of what I use every day, I learned over ten years ago. Computer science never gets old. The notion of indirection is never obsolete. The mental machinery required to navigate up and down in an abstraction hierarchy is universal. Sure, I learn new APIs every day, read about new programming paradigms (hey, functional reactive programming is cool), and so on, but I learned all the basics way back when.
Would you say that all my older experience is worthless? If your hypothesis were correct, I should be able to say that what I'm learning now is more important than what I learned in 1998, and that's just not true.
What are you basing all of these assumptions on? All you've contributed to the conversation is more stereotyping. If you have something to cite I'm happy to look at it.
Most of the assumptions you've made don't apply to me, a 40 year old father of two. It may very well apply to some people similar to me. This is the problem with stereotypes and the problem with people perpetuating them. Oversimplifications are a shoddy mental crutch to deal with the complexities of reality.
Ah yes, 20-somethings without kids are famous for having work be their first priority.
Having kids can put a lot more structure into a life that was missing it before. 20-somethings without kids are usually the ones late in in the morning, missing standups, or the ones working while hung over.
I'm sure that it is possible to get someone who is so dedicated to their work that they have no life at any age. Whether or not this is a good idea for your company is a different question.
There's so much wrong with you condescending and arrogant post that it's hard to know where to begin.
Where do you think bad, old developers come from? Well, they started off as untalented young developers. Right now now there are young developers who are awesome and others who suck and will one day become crappy, old developers.
I could go on about stories where youth and inexperience led to disasters. They would be just as one-sided as talking about old developers who are out of touch. Who do you think it is that's creating all if the tools that you use?
Employees who are parents are less productive. Logically, it follows that there would be measurable degredation in society over time, as long as workers keep procreating. Society's decline is irrefutable.
I hope the explanation for your post is that you're 17. This blatant generalization is idiotic and useless. Reading that reply is a denial of service attack on productive thought.
If your experience of programming becomes worthless after 10 years, then you probably did not focus on the good skills. I don't see how learning how low level hardware works, some time tried languages (C for example), data structures, networking details, debugging protocols of all kinds, structuring programs, working in teams and so on and so forth... could become worthless.
For the idea that parents and older programmers have different priorities, that may be true, but you downplay experience way too much. That said, trying to find 150% dedicated younglings, that don't spend a lot of time on side projects and just enjoying life is not that easy either
This doesn't add up. I started programming in late 80-ies, I've learnt Forth, C, WinAPI (I started seriously developing Windows programs with 2.0! Gosh, how old I am...), then moved to the brighter side, learnt C++, LISP, I used ksh and vi (never was smart enough to master Emacs), used sccs and then rcs, wrote my own implementation of TCP, did some embedded work in C and asm, etc, etc etc.
Given that list, what would you consider obsolete now? It's been 2 decades, not 1, mind you. Asm? Well, the last time I touched this sword for a reason was 2005 and it was SSE assembler, something that compilers were not able to generate efficiently then (and to the large extent still not able).
Since you felt free to generalize us parents, I will feel free to tell you that people without kids are generally way behind us parents in terms of maturity. That might not matter in entry-level jobs, but is quite essential when you climb up a bit.
Somehow, despite having kids, I manage to learn new technologies, even insofar as I was able to write a book about it (Dart in Action). The only reason I had time to do that is because I try to keep my day job between 9-5.
I'm not yet 40, just 35 with 2 kids. I don't agree with any bit of what you wrote.
1. As many people noted - experience is NOT obsolete. The same few concepts re-occur in every software language. You need to learn new APIs and new syntax, but that's total non-sense for good experienced people.
2. People with children have a better understanding of what responsibility is.
3. People with children usually spend less time on dating and going to bars :)
Is it reasonable that we try to build a IT-industry where people can have a family and kids and still be considered productive? Do you plan on having a family and kids yourself one day? Will you change job and start selling bananas that day?
I know of very few other jobs were you are supposed to spend so much of your spare time reading up on new stuff as the IT-business. A few of my friends are physicians and they to spend their spare time reading up on new stuff. Others, not so much.
I don't think this is a sustainable way of doing business in the long run.
The claim about technology moving faster now is absolute garbage. We've had exactly two major pieces of innovation in computer science in the last two decades.
The first was in search and distributed systems, at Google, and the blueprint for their systems was created by two mid-career engineers out of DEC.
The second was the evolution of the smartphone, at Apple, an old company that is known for keeping its engineers for a long ass time.
The most innovative computer language we have was designed in the 50s, and every time I meet an old-time hacker, there's something I can learn from them.
The real lie is that a kid out of college who starts a company is automatically considered an innovator. Rehashing an idea one more time in a slightly more marketable way or creating an app that goes viral has nothing to do with innovation.
In the pop culture age, we constantly confuse the Britney Spears' of the world with real talent, and the Summly's of the world with real innovation.
"If your first priority is your kids, your first priority is not your work. That's fine, it's a choice, but there is a frequent claim that the massive incremental time demand of kids makes one so much more time efficient at work that it more-than-compensates."
I am not young. I've been attending funerals recently, and meeting family members I have not seen for 30+ years. None of us were comparing notes about our productivity.
Insofar as one is the same human being, with the same energy reservoirs and time management skills as before your child was born, this is unlikely.
Your whole post drips with resentment-fueled bigotry and ignorance, making me wonder if you're fighting for attention at your workplace or whatever, feeling unloved. Here's an internet hug. Hugz.
Many years back I -- a new graduate employee -- was chatting with my boss, who was a part owner of the company/president. He had five kids, or maybe even six. He asked me when I was thinking of having children (it was still too early for me, but just as a conversational thing), and my honest answer was that I didn't know how he could afford it.
He then told me about an Arabic parable or the like that each child comes with a bag of money.
That seemed counter intuitive to me, but my life has proven it out. I now have four children, and I would wager good money that I know more current technologies, in much more depth, than you do.
If you have the capacity, having children has a profound ability to make you focus: While I am the same intellectual being, like the vast majority of developers I was absolutely pissing time away before children, and I doubt I passed even 5% productivity. Slashdot was the Reddit of the time, and doing asinine, meaningless implementations for days on ends was just a normal day. And I know this is the case for most developers.
Now I don't have time for the bullshit. I focus specifically on the things that yield success, in the most efficient manner possible. I'm still only maybe 15% productive (still piss away a lot of time), but the result is my own company, a lot of success, etc.
Some of us are 9-5 types without kids. I strongly believe that the work yourself to death model isn't healthy, and in my experience doesn't lead to the best code long term either. I see it all the time from college hires though.
I couldn't agree more. That one phrase was the only thing that bugged me. As if having kids suddenly made me a lower quality coder. I wonder how many of the seasoned veterans that the author lamented losing had kids.
As people get older, they get more experience, which is good. They are also more likely to have a family. Is that supposed to be bad?
I think there are two complementary parts that Linux gets right here. One is as cited in the article, that you get personal glory for improvements you make to the kernel even if they are fairly small. The other part is that there is for Linux someone who will say "no" to a patch. Linus will certainly do it, and other trusted devs will do it too. Linus is perhaps even famous for telling people when they are wrong.
I've seen a number of open-source projects where you get all the personal glory for your additions but there is nobody who takes the responsibility to tell people "no". These projects, almost universally, turn into bloated messes over time. Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games.
> Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games.
I just listened to an episode of Roguelike Radio from the mid-30s (maybe the one with Red Rogue or Thomas Biskup) and Andrew / Keith (IIRC) go on and on about how all the "major" roguelikes are just accretions of features.
This rings very true to me; I grew up playing Nethack [0], and Slash (and then Slash'Em) just got silly with all the things they were adding. Light sabers ? In my roguelike ? Come on, fhgwgads. Nethack at least has a mysterious Dev Team who ostensibly are the guardians of quality... but they _did_ bring Sokoban back from Slash'EM into vanilla.
[0] since Nethack was my first roguelike, I didn't realize until lately just how crappy it is in many ways, most notably that it's very difficult to win before you've figured out all the sources of instadeath (whether you've figured them out by spoilers (written or source-diving) or through (really frickin hard-won) experience).
Try POWDER. It's made practically by one person, and it's very well designed (inspired by Dungeon Crawl Stone Soup). It also has hands down the best writing in any roguelike. God system in particular is unique, all 5 or so gods evaluate your behavior at the same time. Gods like different actions, and whatever you do you will piss off some gods and delight others. It requires some spoilers to understand, but is very fun once you do. POWDER's implementation of Xom is intriguing - he's mentally unstable and changes his preferences every 1000 turns, first he wants you to act like a warrior, second he gets annoyed when you strike a monster in melee. Or even kill anything personally when he impersonates Pax...
But I digress. POWDER feels very much a designed roguelike, not something that evolved.
What's wrong with 9-5? Can't you be passionate about what you work on, excel in you career, yet stick to dedicating ~50% of your time awake to your job?
Unfortunately that kind of thinking is no longer only an American thing. I see it more and more every day. Employers are pushing fresh college grads to work as many hours as possible for a low wage just so they can have a bigger profit. If they burn out, there are others to replace them.
It's amusing watching Americans boast about working past the norm 9-5 hours and calling the rest of the world lazy. My response, "What are you proud about? After working twice as hard, you have the same quality of life as friends in Canada, and statistically speaking you'll drop dead sooner because of stress."
That sounds like an organizational problem to me if you spend 75% of your time at meetings everyday instead of coding (if coding is actually your job).
We can't touch named pipes. Let's add %INTERNAL_NOTIFICATION_SYSTEM%! And let's make it inconsistent with virtually every other named NT primitive.
Linux does this motherfucking bullshit too. "Oh, systemd is piss slow? We'll bung d-bus straight into the kernel, right along side the over 9000 other IPC mechanisms. Everybody uses d-bus these days, it's an Essential System Component. What? Systemd is a crap idea to start with? People like you should be committed."
Besides: you guys have systemd, which if I'm going to treat it the same way I treated NTFS, is an all-devouring octopus monster about crawl out of the sea and eat Tokyo and spit it out as a giant binary logfile.
I keep coming here, writing a bunch of stuff, then removing it.
The author isn't far off. He's over the top on some things, but on the whole it's a good take on some of the reasons Microsoft culture is toxic.
My take:
1. The review system needs to be fixed. They have to stop losing good engineers.
2. They have to somehow get rid of a bunch of toxic people. Many of these are in management. It's hard.
3. They have to stop treating Windows as "the thing we put in all the other things".
Windows is a fine OS; it could be better, and the OP points out a lot of reasons why it isn't. But it's not a great fit for embedded systems, nor game consoles, nor anything where you're paying for the hardware.
But I keep coming back to the review system rewarding the wrong people, causing good people to leave. The underlying backstabbing and cutthroat culture needs to go; it's hurting the bottom line, and I'm surprised the board has been willing to let it happen.
> These junior developers also have a tendency to make improvements to the system by implementing brand-new features instead of improving old ones.
That's exactly the problem that's plaguing Google Chrome right now, although probably for a different reason, as many senior developers still seem to be on board. Google keeps adding new features at high pace and doesn't care what brakes in the process. The amount of unfixed (albeit mostly minor) bugs is huge.
Backward compatibility is an OS performance metric. Maybe sales is too. Microsoft has to think long and hard about any kernel change. In some irony, Microsoft doesn't own the Windows code, and any individual can own the Linux kernel - i.e. Windows lacks forks.
That Microsoft discourages individual junior developers from cowboying, is a point in their favor. Optimization for its own sake is not what benefits their users - real research does.
I thought Microsoft refusing to update their decades behind, ancient C compiler was just to piss me off and make life difficult cross-platform developers that need to work in C. Interesting to see this applies to their own employees too.
I think a lot of people that care about C support are already aware of Herb Sutter's stance on the matter. But knowing why he made his decision doesn't really change anything, its not like C++ never crossed their minds before. And classing it up by calling it official policy doesn't make the pill go down any easier.
Microsoft's position in the market is already big enough that they'll never really be proven wrong on C unless they relent. But they're also not large enough to kill it outright.
tl;dr : corporatism and careerism. It's the death of creativity and productivity. No large organization is immune to it. Never will be. (and yes, that includes Google... it's just much smaller and newer than Microsoft is right now)
I've even seen this mentality in startups. It does have some business rationale, provided you are thinking short term and focused only on near-term goals.
One of the reason businesses have trouble really innovating is that it's hard in a business to work on long-term things when markets are very short sighted. Only mega-corps, monopolies, and governments can usually do that... or hobbyists / lifestyle businesses who are more casual about hard business demands.
That being said, MS is surely cash-rich enough to think long term. So this doesn't apply as much here.
I've also found that of all things optimization almost gets you looked down upon in most teams -- even young ones. "Premature optimization is the root of all evil," and all that, which is usually misinterpreted as "optimization is naive and a waste of time." It's seen as indicative of an amateur or someone who isn't goal-focused. If you comment "optimized X" in a commit, you're likely to get mocked or reprimanded.
In reality, "premature optimization is the root of all evil" is advice given to new programmers so they don't waste time dinking around with micro-optimizations instead of thinking about algorithms, data structures, and higher order reasoning. (Or worse, muddying their code up to make it "fast.") Good optimization is actually a high-skill thing. It requires deep knowledge of internals, ability to really comprehend profiling, and precisely the kind of higher-order algorithmic reasoning you want in good developers. Most good optimizations are algorithmic improvements, not micro-optimizations. Even good micro-optimization requires deep knowledge-- like understanding how pipelines and branch prediction and caches work. To micro-optimize well you've got to understand soup-to-nuts everything that happens when your code is compiled and run.
Personally I think speed is really important. As a customer I know that slow sites, slow apps, and slow server code can be a reason for me to stop using a product. Even if the speed difference doesn't impact things much, a faster "smoother" piece of code will convey a sense of quality. Slow code that kerchunks around "feels" inferior, like I can see the awful mess it must be inside. It's sort of like how luxury car engines are expected to "purr."
An example: before I learned it and realized what an innovative paradigm shift it was, speed is what sold me on git. The first time I did a git merge on a huge project I was like "whoa, it's done already?" SVN would have been kerchunking forever. It wasn't that the speed mattered that much. It was that the speed communicated to me "this thing is the product of a very good programmer who took their craft very seriously as they wrote it." It told me to expect quality.
Another example: I tried Google Drive, but uninstalled it after a day. It used too much CPU. In this case it actually mattered -- on a laptop this shortens battery life and my battery life noticeably declined. This was a while ago, but I have not been motivated to try it again. The slowness told me "this was a quick hack, not a priority." I use DropBox because their client barely uses the CPU at all, even when I modify a lot of files. Google Drive gives me more storage, but I'm not content to sacrifice an hour of battery life for that.
(Side note: on mobile devices, CPU efficiency has a much more rigid cost function. Each cycle costs battery.)
Speed is a stealth attribute too. Customers will almost never bring it up in a survey or a focus group unless it impacts their business. So it never becomes a business priority.
> In reality, "premature optimization is the root of all evil" is advice given to new programmers so they don't waste time dinking around with micro-optimizations instead of thinking about algorithms, data structures, and higher order reasoning. (Or worse, muddying their code up to make it "fast.")
It's also for experienced programmers who dink around with macro-optimizations. For example, designing an entire application to be serializable-multi-threaded-contract-based when there's only a handful of calls going through the system. Or creating an abstract-database-driven-xml-based UI framework to automate the creation of tabular data when you have under a dozen tables in the application.
premature optimization is the root of all evil is a really really important mindset, and I agree it doesn't mean to not optimize, and many developers seem to take it that way.
X+1 = How many transactions your business does today
Y = How many transactions your business needs to do in order to survive
Y/X = What the current application needs to scale to in order to simply survive. This is the number where people start receiving paychecks.
(Y/X)4 = How far the current application needs to scale in order to grow.
The goal should be to build an application that can just barely reach (Y/X)4 - this means building unit tests that test the application under a load of (Y/X)4 and optimizing for (Y/X)4
Spending time trying to reach (Y/X)20 or (Y/X)100 is what I'd call premature optimization.
Disclaimer: (Y/X)4 is no real point of data that I know of, just something I pulled out as an example, anyone who knows of actual metrics used please feel free to correct.
> > In reality, "premature optimization is the root of all evil" is advice given to new programmers
> It's also for experienced programmers who dink around with macro-optimizations.
The hilarious thing is that everyone seems to think everyone else's optimizations are premature.
Don't believe me? Just go ahead and ask someone an optimization question, like maybe on StackOverflow.
Those who respond might not know you, but I'll be damned if their first reply isn't to tell you that you probably shouldn't even be optimizing in the first place, because obviously if you'd thought about it more carefully, you'd have realized it's premature.
"Or creating an abstract-database-driven-xml-based UI framework to automate the creation of tabular data when you have under a dozen tables in the application."
I'm not an experienced programmer, and I'd use that approach anyway, simply because it makes sense (is there any alternative that isn't worse?). What does that mean? (Oh, and I'd skip XML. I abhor violence.)
i admit to doing this... made a multithreaded simulation framework for use by my game team.... too bad the games they make with are so computationally simple as to make the sim framework a complete waste :(
The concept of "premature optimization" also has another connotation in product development: Don't waste too much time making that product or feature optimized until you are convinced you can actually sell it. It's not that optimization is bad, but optimization before market trial (premature) can result in you spending precious time working hard on the wrong thing.
Optimizing the right thing is good, but figure out what that thing is first.
In a publicly traded company, such as Microsoft, that doesn't take long. It is shareholder value. It is not Optimization "for its own sake" promoted in the article. I savvy developer made need to run business tests against their software.
I assume some other dev with source access at Microsoft could verify that fact, broadcast that the fact is true, and thereby stamp the OP as actually who he says he is.
From what I've heard (unofficially) the source is typically watermarked to identify the sources of any leaks. He may have given himself away with the SHA-1.
I doubt it. Not only would that involve generating a different watermark for each person with access to the code, but in doing so you also raise complications for source control. Additionally, it is almost impossible for developers not to notice if supposedly identical files are not identical, not to mention the fact that implementing the type of system without the conscious corporation of the developers seems impossible.
I don't know what the author was thinking of specifically, but the OOM killer seems like a bit of a hack to me. I can think of situations where I'd rather have an allocation fail upfront predictably than be randomly killed by the system.
A lot of people seem to think you're screwed when you're out of memory and so think killing the process is acceptable, but I've worked on code bases where it's actually handled somewhat gracefully. (Although, on NT, if you run out of nonpaged pool you start to get weird, random I/O failures.)
Wikipedia tells me that Windows NT was started in 1988 and shipped in 1993, while Linux's first release was in 1991, giving them approximately the same age. Furthermore, NT shipped with UCS-2 support, not UTF-16, as UTF-16 did not exist until 1996. UTF-16 was a migration that they completed in Windows 2000. UTF-8 was first presented in 1993 and is, therefore, older than UTF-16.
With all of these facts being true (I hope), today things are such that the Windows world mostly (but not completely -- they sometimes assume two-byte characters) correctly implements UTF-16, while the Linux world correctly implements UTF-8.
"Unicode-enabled functions are described in Conventions for Function Prototypes. These functions use UTF-16 (wide character) encoding, which is the most common encoding of Unicode and the one used for native Unicode encoding on Windows operating systems. Each code value is 16 bits wide .. New Windows applications should use UTF-16 as their internal data representation."
There is a great quote in the post: "Incremental improvements just annoy people and are, at best, neutral for your career."
Working for several different Fortune 500 companies this has been my experience many times. I have worked on products where I made a large number of incremental improvements that never got incorporated into the released product. Taken on their own the improvements don't change things much, but in the aggregate they make a significantly better product. Why didn't most of my incremental improvements get incorporated into the product? Every change, no matter how minor, is perceived by management to be a highly risky proposition. Product managers won't get promoted because the product is 4% better, but they could potentially get fired if that same incremental improvement had an unforeseen negative consequence.
Once the short term sales mindset gets power, hit products are viewed as cows to be milked and not as things that need continued maintenance and improvement. As an engineer in such an enterprise you watch helplessly as your competitors slowly but steadily pass you, and then someday upper management wonders why the magic money machine broke down. Some large enterprises avoid this fate but they seem to be the exception.
The problems with the NT kernel are nothing new. When I suggested for Windows to use a BSD kernel to overcome its current limitations, I got lots of disagreement. Microsoft developers seem to be happy with the current state and have no vision of a totally new architecture.
People are praising BSD like it was all hot and new. OS X is built on a BSD which has it's roots in 60's and 70's OS design, just like the VMS roots of WinNT.
OS X didn't change the world by bringing some great new underlying architecture to the table. In fact, their kernel and filesystem are arguably getting long in the tooth. The value that OS X brought to the table was the fantastic Carbon and Cocoa development platforms. And they have continued to execute and iterate on these platforms, providing the "Core" series of APIs (CoreGraphics, CoreAnimation, CoreAudio, etc.) to make certain HW services more accessible.
There's very little cool stuff to be gained in the windows world by developing a new kernel from scratch. A quantum leap would not solve MS's problem. The problem is the platform. What's really dead and bloated is the Win32 subsystem. The kernel doesn't need major tweaking. In fact, the NT kernel was designed from the beginning such that it could easily run the old busted Win32 subsystem alongside a new subsystem without needing to resort to expensive virtualization.
Unfortunately, the way Microsoft is built today it have a fatal organizational flaw that prevents creating the next great Windows platform. The platform/dev tools team and the OS team are in completely different business groups within the company. The platform team develops the wonderful .NET platform for small/medium applications and server apps while the OS team keeps crudging along with Win32. Managed languages have their place, but they have yet to gain traction for any top shelf large-scale windows client application vendors (Adobe, even Microsoft Office itself, etc.) Major client application development still relies on unmanaged APIs, and IMHO the Windows unmanaged APIs are arguably the worst (viable) development platform available today.
What Windows needs is a new subsystem/development platform to break with Win32, providing simplified, extensible unmanaged application development, with modern easy-to-use abstractions for hardware services such as graphics, data, audio and networking. This is starting to come to fruition with WinRT, but the inertia in large scale apps is unbelievable.
The NT-based kernels were not as big a problem for Microsoft as CE-based OSs were. Nokia transitioned away from a Linux-based smartphone OS to Windows Phone 7, based on Windows Embedded CE 6. It is clear that Windows Phone 7 was uncompetitive with other smartphone OSs. That put Nokia in limbo for a year when they could least afford it.
Now the problem is not so much the kernel, but all the cruft a Windows-based OS drags along with it. The most visible symptom is the amount of storage used up by stuff that adds little or no readily apparent value to Windows RT.
Windows OSs are heavy compared to Android and other Linux-based OSs. That means inexpensive tablets and phones won't run Windows, and Windows is shut out of mobile device markets in India, Africa, and other developing areas. These markets are how Yulong (who??!!) became a top 10 mobile OEM.
As an outsider, it looks like MSFT does not grasp kernel performance as a super important feature of Windows. They add so much bells & whistles over the UI, but the inside sucks release after release
As someone else posted in this same thread: is modern Windows substantially slower than the OSes it competes with (in anything other than a few uncommon niches)? If it's more than a few percentage points slower for anything but rare scenarios, I'd be surprised.
The description sounds accurate compared to what I heard when I was in contact with Microsoft insiders. That was about 15 years ago. What's surprising to me is that it is still this way.
A lot of the justification for being exceedingly conservative about performance improvements was based on the difficulty of doing testing that is sufficient to leave no doubt that changes don't cause regressions nor any side effects bearing on compatibility.
I'm surprised that hasn't changed, or that the value equation about possibly breaking customer software hasn't changed. By now this approach must be a very entrenched culture.
I think the main performance problem with NT is its scheduler. Compared to unix-type schedulers, it doesn't give very good interactive performance when there are cpu-bound processes. There have been a lot of hacks over the years (like throwing cpu-starved processes a bone when they haven't had any action in 4 seconds), but unix/linux just has a better designed scheduler that gives better performance in most situations.
The Linux kernel developers take backwards compatibility very seriously. Here is a tame excerpt from Linux on the subject:
"Seriously. Binary compatibility is so important that I do not want
to have anything to do with kernel developers who don't understand
that importance. If you continue to pooh-pooh the issue, you only show
yourself to be unreliable. Don't do it." [1]
Why is it that I can run my Radeon x1900 under Windows 7/8 with Windows Vista drivers? Runs StarCraft2 and a bunch of newer FPS games just fine.
Linux on the other hand is a disaster. The kernel devs are so determined to break binary compatibility, I haven't been able to run with ATI's proprietary binary drivers for years. While AMD was a good open source citizen and released the specs, the open source drivers for my card are useless for anything other than 2D.
I support conservative engineering for file systems.
NTFS is highly reliable and Microsoft wants to build on that success with ReFS. In years of working with NTFS I've never had a serious filesystem wreck (although I did have one that was never quite the same after I made a few hundred million small files)
The NTFS story isn't that different from the ext(2|3|4) story on Linux. ext-N filesystems are boring and not particularly fast, but they're reliable. I've tried other filesystems on Linux but have often had wrecks or noticed data corruption.
Look at it this way: you're lined up to work in a company with far more capability and potential than almost every other western organisation. When you're compared to the likes of Google and Apple as examples of where you might be lacking, you're doing good.
Besides, even if everything is as bad as they say and you absolutely cannot work in that environment, interning is the perfect time to discover that without as significant commitments. It will also prove to be a valuable lesson that you can then take to other companies.
"our good people keep retiring or moving to other large technology companies, and there are few new people achieving the level of technical virtuosity needed to replace the people who leave."
in the post PC era Kernels are not as hot as they were in 2000
The problems with the NT kernel are nothing new. When I suggested for Windows to use a BSD kernel to overcome its current limitations, I got lots of disagreement. Microsoft developers seem to be happy with the current state and have no vision of a totally new architecture.
Sad post IMO, which will unfortunately and unduly tarnish Microsoft's reputation. I would suppose it was written by a developer who has just become senior enough to see some of what's going on, but has not yet realized it's everywhere like that. One day, he'll understand that those challenges too need to be managed.
It's nice to see that internal developers feel the same way about XNA that external developers (who used to build XNA games, or still build XNA games) do.
From the outside I always assumed the constant flood of new, half-baked features instead of fixes and improvements to old ones was caused by interns and junior devs looking for glory - sad to hear that's actually partly true. I always considered frameworks like WPF (or Flex, for that matter) 'intern code' - not that interns necessarily wrote them, but they reek of not-experienced-enough engineers trying to solve problems by writing a bunch of new code, instead of fixing existing code.
It really is too bad, though. There are parts of the NT kernel (and even the Win32 API) that I consider a joy to use - I love IOCP, despite its warts, and APIs like MsgWaitForMultipleObjects are great tools for building higher-level primitives.
Plus, say what you want about GDI (there's a lot wrong with it at this point), but it's still a surprisingly efficient and flexible way to do 2D rendering, despite the fact that parts of it date back to before Windows 3.1. Some really smart people did some really good API design over time over at Microsoft...
Actually, I think one NT's largest advantages over POSIX systems is process management: yes, the venerable CreateProcess API.
See, in Windows, processes are first class kernel objects. You have handles (read: file descriptors) that refer to them. Processes have POSIX-style PIDs too, but you don't use a PID to manipulate a process the way you would with kill(2): you use a PID to open a handle to a process, then you manipulate the process using the handle.
This approach, at a stroke, solves all the wait, wait3, wait4, SIGCHLD, etc. problems that plague Unixish systems to this day. (Oh, and while you have a handle to a process open, its process ID won't be re-used.)
It's as if we live in a better, alternate universe where fork(2) returns a file descriptor.
You can wait on process handles (the handle becomes signaled and the wait completes when the process exits). You can perform this waiting using the same functions you use to wait on anything else, and you can use WaitForMultipleObjects as a kind of super-select to wait on anything.
If you want to wait on a socket, a process, and a global mutex and wake up when any of these things becomes available, you can do that. The Unix APIs for doing the same thing are a mess. Don't even get me started on SysV IPC.
Another thing I really like about NT is job objects (http://msdn.microsoft.com/en-us/library/windows/desktop/ms68...). They're a bit like cgroups, but a bit simpler (IMHO) to set up and use.
You can apply memory use, scheduling, UI, and other restrictions to processes in job objects. Most conveniently of all, you can arrange for the OS to kill everything in a job object if the last handle to that job dies --- the closest Linux has is PR_SET_PDEATHSIG, which needs to be set up individually for each child and which doesn't work for setuid children.
(Oh, and you can arrange for job objects to send notifications to IO completion ports.)
Yes, Windows gets a lot wrong, but it gets a lot right.
> WaitForMultipleObjects as a kind of super-select to wait on anything
Except for the 64 handle limit, which makes it largely useless for anything that involves server applications where the number of handles grows with the number of clients. So then you'd spawn "worker" threads, each handling just 64 handles. And that's exactly where your code starts to go sour - you are forced to add fluff the sole purpose of which is to work around API limitations. What good is the super-select if I need cruft to actually use it in the app?
And don't get me started on the clusterfuck of the API that is IOCP.
> Yes, Windows gets a lot wrong, but it gets a lot right.
Oh, no, it doesn't.
Specifically, what Windows does not get is that it's other developers that are using its services and APIs, not just MSSQL team that can peek at the source code and cook up a magic combination of function arguments that does not return undocumented 0x8Fuck0ff. I have coded extensively for both Linux and Windows, including drivers and various kernel components, and while Windows may get things right at the conceptual level, using what they end up implemented as is an inferior and painful experience.
34 replies →
My personal annoyance is the introduction of win32k in NT4. With per-session CSRSS existing since WinFrame and Vista+ having Session 0 Isolation, not to mention much faster processors, and the removal of XPDM and the requirement of the DWM in Win8, it doesn't make as much sense anymore.
Update: And I forgot to mention the font parsing security issues (look up Duqu), and the issues with user mode callbacks too.
16 replies →
Note that the problem you describe for process is not a fundamental design choice. For instance, FreeBSD has added such API.
http://www.freebsd.org/cgi/man.cgi?query=pdfork
2 replies →
IO Completion Ports are a pretty nice feature too, in an of themselves.
Great explanation
> I always considered frameworks like WPF (or Flex, for that matter) 'intern code' - not that interns necessarily wrote them, but they reek of not-experienced-enough engineers trying to solve problems by writing a bunch of new code, instead of fixing existing code.
This is unfair and unfounded accusation. If you look at the time frame, all the major platform were working on their first hardware accelerated UI toolkits and went through the similar teething problems (Cocoa anyone?). WinForms was a dead end, there was no fixing to do. WPF has turned out well enough, and WinRT has evolved into something very efficient (e.g. by using ref counting rather than GC).
> Plus, say what you want about GDI (there's a lot wrong with it at this point), but it's still a surprisingly efficient and flexible way to do 2D rendering
Not anymore. I get it that some love to use antique APIs and computer systems just for the sake of being retro, but when every computer ships these days with a GPU, using GDI is not even close to pragmatic.
> ...WinRT has evolved into something very efficient (e.g. by using ref counting rather than GC).
Ah you mean by causing cache stalls when doing increments/decrements after each assignment and allowing for cyclic references?
17 replies →
GDI has been GPU accelerated literally forever. Vista may have dropped hardware acceleration for GDI, but it was promptly brought back in Windows 7.
Since the Win32 UI stack uses GDI, it was hardware accelerated before WPF even existed.
1 reply →
The "Inside Windows Kernel" book series are quite interesting to understand on how it all works, and even some of the initial VMS influence in the original kernel design.
> "It's nice to see that internal developers feel the same way about XNA that external developers (who used to build XNA games, or still build XNA games) do. From the outside I always assumed the constant flood of new, half-baked features instead of fixes and improvements to old ones was caused by interns and junior devs looking for glory ..."
Am I understanding you correct; are you implying that you think XNA was created by juniors? Would you say that because of this, it's a good thing XNA is being killed?
Cause personally I think it's been a terrible decision by Microsoft to kill XNA. A lot of indie game developers have relied on XNA and I really feel Microsoft can use the indie support. Sure, big name games might be a priority, but personally I feel most _interesting_ work is being done by indies. Indies tend to be less concerned with proven formulas and seem to see it more of a creative outlet for themselves[1]. I think it's a good thing frameworks like Monogame[2] exist, so developers can still put their existing XNA knowledge to good use - and not just limited to the Windows platform, but e.g. iOS and Android as well.
The Monogame website might not show very impressing examples, but a game like Bastion[3] was ported to other platforms using Monogame, showing very high-quality work can be created with XNA.
[1]: http://www.youtube.com/watch?v=GhaT78i1x2M
[2]: http://monogame.codeplex.com
[3]: http://supergiantgames.com/index.php/2012/08/bastions-open-s...
What is the feeling about XNA? I haven't followed the area, so I found it unfortunate that it was treated in the original post without explanation. Was XNA a good thing or a bad thing? Why?
It was loved by Indies for XBox 360 development, got so popular, Microsoft decided to make it the way to develop games on the Windows Phone 7.
When Windows Phone 8 came out, the native renaissance was full speed inside Microsoft, C++ with DirectX became the official way and XNA was canned while Microsoft left MonoGame and Unity pick up the C# developers still willing to invest into the platform.
The sad thing is that this is not a native vs managed issue, because even C# gets compiled to native code when targeting Windows Phone 8. Just plain product management decision to kill the product, which was being sold as the way to develop Windows Phone games.
9 replies →
And what is in your opinion better then WPF?
> Another reason for the quality gap is that that we've been having trouble keeping talented people. Google and other large Seattle-area companies keep poaching our best, most experienced developers, and we hire youths straight from college to replace them.
I will say all of the ex-Microsoft folks I've encountered at Google Seattle have been fantastic.
On a related note, it's stupidly easy to get code accepted by another team at Google.
Also we're hiring.
Too bad you guys don't seem to give a second glance at someone without any formal education, despite working on systems for government, training, banking security and airline industries.
I have zero formal education past A levels, but I've had two in-person interviews at Google and had an offer, and I have their charming recruiters chasing me to make sure I wouldn't be happier working for Google every 6 months or so. :)
It does take a ferociously long time to get through the initial screening if you just send them a CV, and I'm sure a lot of people fall through the cracks - my suggestion would be to find a Googler or Xoogler and ask them to recommend you, I understand you get a lot more traction that way. Chin up.
"Too bad you guys don't seem to give a second glance at someone without any formal education, despite working on systems for government, training, banking security and airline industries."
I find it doubtful that your lack of education alone is what's keeping you from getting a callback. Do you have a portfolio linked from your resume? Have you got friends that work for these companies that are willing to go over your resume with you?
If you know anyone internal who's familiar with your work, have them write up a referral for you. That can give the recruiters a lot more context.
To be fair, and in context: neither does Microsoft.
1 reply →
>Also we're hiring.
Insult to injury? ;)
> On a related note, it's stupidly easy to get code accepted by another team at Google.
Unless that other team is Android. Though then you could submit to AOSP directly (assuming the issue you are addressing wasn't fixed internally 4 months ago, but how would you know?).
> Also we're hiring.
Bah... my friends keep referring me, but recruiters never seem to like it... ;-)
I look forward to further entanglement with a hiring process much more toxic than Microsoft's. Plus the joy of being hired for a mystery job where all my Big O knowledge goes to writing scripts for placing free porn ads.
I actually enjoy working at Microsoft --- I'm in Phone, working on telemetry and various other things --- and I've met a ton of very smart people people. I've also made the cross-team (and cross-org) contributions that the OP says are nearly impossible (just today, even). While the OP makes a few good points (some teams are kinda reluctant to take patches), I think he's grossly exaggerating the problems, and the level of vitriol really isn't called for either.
He's also slightly off-base on some of the technical criticisms: there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent. Besides, change _does_ happen: Arun Kishan (who is smarter than I'll ever be) broke the dispatcher lock a while ago (see http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewe...) when nobody thought it could be done.
By the way: there's actually quite a lot of information available on Windows Internals. In fact, there's a whole book, Windows Internals (http://technet.microsoft.com/en-us/sysinternals/bb963901.asp...) on exactly how the NT kernel and related system components work. The book is highly recommended reading and demystifies a lot of the odder parts of our API surface. We use it internally all the time, and reading it, you'll repeatedly say to yourself, "Ah, so that's why Foo does Bar and not Qux! Yeah, that's a good reason. I didn't think of it that way."
"there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent."
One of the issues the author seemed to be alluding to is that the loss of experienced developers makes it really hard to keep track of "line 123 of foo_dispatch.c does this because of vital business reason xyz or compatibility reason uvw" and "line 123 of foo_dispatch.c was written while I was hungover and trying to make a deadline--it looks clever, but feel free to junk it.".
This issue is compounded when you are hiring safe, and you have a culture where making gross fuckups while trying to make progress is discouraged. It is neither good nor bad--after all, I too enjoy using stable software--but there is a price to be paid if devs don't feel comfortable making breaking changes.
There are always good reasons for doing everything. But this is how you end up with a "death by a thousand cuts" scenario. Sometimes, if you want to be great, you have to be bold. Windows NT was bold back in 1995 when I first played with 3.51. It was the best OS out there by far.
Now, once you set up a system where people play it safe more than they innovate, you'll get a situation where all of the best programmers will want to leave because it's boring, and you're left with mediocre programmers who aren't good enough to be bold and innovative. This is exactly what the OP describes.
Do you think the Phone division is working differently to the rest of the organisation? Windows Phone 7+ really wowed me. Its a great product.
Look at recent Microsoft releases: we don't fix old features, but accrete new ones. New features help much more at review time than improvements to old ones.
(That's literally the explanation for PowerShell. Many of us wanted to improve cmd.exe, but couldn't.)
Ahh, I was wondering about that. So, I guess I'll just keep using cygwin.
On another note, I recently asked a friend who works at Microsoft how work is going. His reply: "Well, it's calibration time, so lots of sucking up to the boss." Must be hard to get much actual work done when you're worried about that all the time.
For those unfamiliar with what "calibration" is in Microsoft: http://minimsft.blogspot.com/2011/04/microsofts-new-review-a...
Powershell is so damn good and is probably so because it was not improvement of cmd. Give credit where it is due.
I'm always found it difficult to read and impossible to debug.
It's useful, sure, but nice? I'm not convinced.
How about an empty recycle bin command? Why, certainly, you can do that:
Teh F33243ck. Readable much? I sure hope your powershell scripts come with comments; the one's we've inherited sure didn't and they're black magic.
6 replies →
I really can't agree with that. To me, Powershell feels like it was designed by geniuses and implemented by interns.
Absolutely basic stuff, like "find the path of this file" (%~dp0 in batch) is a 5 line copy-paste. Why? It's lovely that I can pipe .NET objects places, but every time i want it to actually do something, I end up in copy-paste hell.
3 replies →
I don't understand why they felt the need to defy standards with powershell. It's not similar to anything else i'm familiar with. All they had to do was copy bash and port some of the GNU tools.
Would be better if people actually used it!
1 reply →
As an ex-MS employee I always feel obligated to point out that MS has well over 50,000 employees spread out across countless teams and divisions. One employee's experience is never indicative of the company as a whole. I never went through "calibration time" while at MS or anything even close to it.
Yep, sounds about right. Building new stuff is rewarded far more than fixing broken shit. Do you want to make it right or do you want to get promoted?
michaelochurch has talked about problems with this, performance reviews and closed allocation before.
well, he doesn't talk about anything else :)
I do wonder, however, whether 'boo! MS's rating system is suboptimal, to say the least!' is a very useful comment here (which might by why we haven't seen Michael write here yet)
Yeah, and that was about Google, which has a much better reputation than Microsoft in this area. I wonder how he would like working at MS. I doubt he'd last a week there!
It's possible he deleted his post because he realized that he included something in it that could identify him, which could be bad for his career.
It might have been better to rewrite his post into your own words, and take out some of the unnecessary detail, rather than literally repost.
I'd agree. I know the OP is trying to spread knowledge and this is a great read, but I think this was slightly rude, not to mention questionable fair use.
There's no "fair use" argument at all. Copying an entire work, then distributing it with no transformative change or criticism, is inexcusable infringement should the author want to enforce his rights.
2 replies →
"His post has been deleted! Why the censorship?"
We didn't delete it. It was deleted by whoever posted it.
Thanks. I added the explanation.
To be fair to pg, I feel like this was the obvious conclusion (that an anonymous user posting via tor would redact)
No delete button on the internet v125786123. Sidebar: do deleted comments appear with showdead on, or just ones that are deleted on HN side.
>do [deleted-by-author] comments appear with showdead on[?]
No, they do not. Specifically, I have showdead on, and when trying to view the comment in question, I see only "[deleted]".
>We fill headcount with nine-to-five-with-kids types,
then
>We occasionally get good people anyway
Uh... nine-to-five-with-kids type here. Thanks for the stereotyping... from your safe corporate nest.
Otherwise an insightful post.
If your first priority is your kids, your first priority is not your work. That's fine, it's a choice, but there is a frequent claim that the massive incremental time demand of kids makes one so much more time efficient at work that it more-than-compensates. Insofar as one is the same human being, with the same energy reservoirs and time management skills as before your child was born, this is unlikely.
I understand the reason this fiction is maintained: people with kids need the job even more than people without, and have an interest in denouncing people who claim kids make you less productive. There's also the second order effect in which "with kids" is correlated with "older".
The net of it though is that devs with kids tend to assign work a lower priority, to take fewer risks, to need more money, and to be older and hence less familiar with new technologies (and too busy to learn in their free time).
Society doesn't have a good answer for this situation yet. In times past, technology didn't move so fast that experience was mostly obsolete (and hence useless) in a decade's time. A 40 year old farmer with kids in 1713 would probably have much to teach a young whippersnapper. The same isn't true for a 40 year old programmer with kids in 2013.
LOL at technology moving fast and experience becoming obsolete in "a decade's time." I'm not even that old--I started programming in the late 1990's--but I've been programming more than a decade, and as far as I can tell all that has happened to mainstream programming technology in that time span is bike-shedding. There has been basically nothing new in mainstream programming since about the 1980's ("Oh, NewtonScript looks like C now instead of Pascal, and we're calling it Javascript? That's nice.") In many ways it's gone backwards (the sorry state of Javascript IDE's relative to Visual Studio 6.x not to mention Smalltalk IDE's a decade older than even that). "Oh, you can use threads in web apps now? Like you could in Win32 apps since 1995?"
To a first approximation, all this new "web technology" is just a way of doing what you could've done on an internet-connected NeXT machine 20 years ago, only more brain-damaged and frustrating to work with.
The only place technology has advanced in that time period is in domain-specific areas (which of course requires experts, not young whippersnappers), and infrastructure (the world looks a lot different for computers with pervasive 4G--the software is just an adaptation of proven concepts to uses enabled by innovative hardware).
Programming is applied mathematics, and that hasn't changed substantially since people were developing software on punch cards.
9 replies →
When will we get it that in real life, people don't have "first priorities" just as they don't have "arch enemies"? :) Really, any sane not-overworked not-burned-out team will benefit from older more experienced devs in it. If you're overworking the shit out of everybody, of course that the older-with-kids guys will underperform. But if you are overworking people like hell, it probably just means that you are doing things very inefficiently and compensating by "sheer work power" and shooting yourself in the foot by making the experience holders work at 40% of their top performance because of the environment (ok, I agree, this might be a good trade-off for some businesses). Chill out a bit, do things the right way and at the "natural" pace and you don't need to care about this "problem" or use the nine-to-five-with-kids stereotype. As one wise man told me once: you can't "move fast and break things" kiddo, because all that time spent fixing the things you break actually means that you're not moving as fast as you can or fast enough.
> experience was mostly obsolete (and hence useless) in a decade's time
I'm 28, which means I'm young enough that even people like you believe I can learn new tricks. As far as I can tell, my brain is still fully functional. That said, most of what I use every day, I learned over ten years ago. Computer science never gets old. The notion of indirection is never obsolete. The mental machinery required to navigate up and down in an abstraction hierarchy is universal. Sure, I learn new APIs every day, read about new programming paradigms (hey, functional reactive programming is cool), and so on, but I learned all the basics way back when.
Would you say that all my older experience is worthless? If your hypothesis were correct, I should be able to say that what I'm learning now is more important than what I learned in 1998, and that's just not true.
6 replies →
What are you basing all of these assumptions on? All you've contributed to the conversation is more stereotyping. If you have something to cite I'm happy to look at it.
Most of the assumptions you've made don't apply to me, a 40 year old father of two. It may very well apply to some people similar to me. This is the problem with stereotypes and the problem with people perpetuating them. Oversimplifications are a shoddy mental crutch to deal with the complexities of reality.
4 replies →
Ah yes, 20-somethings without kids are famous for having work be their first priority.
Having kids can put a lot more structure into a life that was missing it before. 20-somethings without kids are usually the ones late in in the morning, missing standups, or the ones working while hung over.
I'm sure that it is possible to get someone who is so dedicated to their work that they have no life at any age. Whether or not this is a good idea for your company is a different question.
There's so much wrong with you condescending and arrogant post that it's hard to know where to begin.
Where do you think bad, old developers come from? Well, they started off as untalented young developers. Right now now there are young developers who are awesome and others who suck and will one day become crappy, old developers.
I could go on about stories where youth and inexperience led to disasters. They would be just as one-sided as talking about old developers who are out of touch. Who do you think it is that's creating all if the tools that you use?
It's about the work.
Employees who are parents are less productive. Logically, it follows that there would be measurable degredation in society over time, as long as workers keep procreating. Society's decline is irrefutable.
I hope the explanation for your post is that you're 17. This blatant generalization is idiotic and useless. Reading that reply is a denial of service attack on productive thought.
3 replies →
If your experience of programming becomes worthless after 10 years, then you probably did not focus on the good skills. I don't see how learning how low level hardware works, some time tried languages (C for example), data structures, networking details, debugging protocols of all kinds, structuring programs, working in teams and so on and so forth... could become worthless.
For the idea that parents and older programmers have different priorities, that may be true, but you downplay experience way too much. That said, trying to find 150% dedicated younglings, that don't spend a lot of time on side projects and just enjoying life is not that easy either
1 reply →
This doesn't add up. I started programming in late 80-ies, I've learnt Forth, C, WinAPI (I started seriously developing Windows programs with 2.0! Gosh, how old I am...), then moved to the brighter side, learnt C++, LISP, I used ksh and vi (never was smart enough to master Emacs), used sccs and then rcs, wrote my own implementation of TCP, did some embedded work in C and asm, etc, etc etc.
Given that list, what would you consider obsolete now? It's been 2 decades, not 1, mind you. Asm? Well, the last time I touched this sword for a reason was 2005 and it was SSE assembler, something that compilers were not able to generate efficiently then (and to the large extent still not able).
3 replies →
Wow. One of the most offensive, patronising, arrogant, ignorant posts I've seen here.
Were you parents rubbish slackers because they made the mistake of having you? Or did they work so hard they forgot to bring you up well?
Since you felt free to generalize us parents, I will feel free to tell you that people without kids are generally way behind us parents in terms of maturity. That might not matter in entry-level jobs, but is quite essential when you climb up a bit.
2 replies →
"too busy to learn in their free time"
Somehow, despite having kids, I manage to learn new technologies, even insofar as I was able to write a book about it (Dart in Action). The only reason I had time to do that is because I try to keep my day job between 9-5.
I'm not yet 40, just 35 with 2 kids. I don't agree with any bit of what you wrote. 1. As many people noted - experience is NOT obsolete. The same few concepts re-occur in every software language. You need to learn new APIs and new syntax, but that's total non-sense for good experienced people.
2. People with children have a better understanding of what responsibility is.
3. People with children usually spend less time on dating and going to bars :)
4. We tend to be more time efficient
Is it reasonable that we try to build a IT-industry where people can have a family and kids and still be considered productive? Do you plan on having a family and kids yourself one day? Will you change job and start selling bananas that day?
I know of very few other jobs were you are supposed to spend so much of your spare time reading up on new stuff as the IT-business. A few of my friends are physicians and they to spend their spare time reading up on new stuff. Others, not so much.
I don't think this is a sustainable way of doing business in the long run.
The claim about technology moving faster now is absolute garbage. We've had exactly two major pieces of innovation in computer science in the last two decades.
The first was in search and distributed systems, at Google, and the blueprint for their systems was created by two mid-career engineers out of DEC.
The second was the evolution of the smartphone, at Apple, an old company that is known for keeping its engineers for a long ass time.
The most innovative computer language we have was designed in the 50s, and every time I meet an old-time hacker, there's something I can learn from them.
The real lie is that a kid out of college who starts a company is automatically considered an innovator. Rehashing an idea one more time in a slightly more marketable way or creating an app that goes viral has nothing to do with innovation.
In the pop culture age, we constantly confuse the Britney Spears' of the world with real talent, and the Summly's of the world with real innovation.
"If your first priority is your kids, your first priority is not your work. That's fine, it's a choice, but there is a frequent claim that the massive incremental time demand of kids makes one so much more time efficient at work that it more-than-compensates."
I am not young. I've been attending funerals recently, and meeting family members I have not seen for 30+ years. None of us were comparing notes about our productivity.
Insofar as one is the same human being, with the same energy reservoirs and time management skills as before your child was born, this is unlikely.
Your whole post drips with resentment-fueled bigotry and ignorance, making me wonder if you're fighting for attention at your workplace or whatever, feeling unloved. Here's an internet hug. Hugz.
Many years back I -- a new graduate employee -- was chatting with my boss, who was a part owner of the company/president. He had five kids, or maybe even six. He asked me when I was thinking of having children (it was still too early for me, but just as a conversational thing), and my honest answer was that I didn't know how he could afford it.
He then told me about an Arabic parable or the like that each child comes with a bag of money.
That seemed counter intuitive to me, but my life has proven it out. I now have four children, and I would wager good money that I know more current technologies, in much more depth, than you do.
If you have the capacity, having children has a profound ability to make you focus: While I am the same intellectual being, like the vast majority of developers I was absolutely pissing time away before children, and I doubt I passed even 5% productivity. Slashdot was the Reddit of the time, and doing asinine, meaningless implementations for days on ends was just a normal day. And I know this is the case for most developers.
Now I don't have time for the bullshit. I focus specifically on the things that yield success, in the most efficient manner possible. I'm still only maybe 15% productive (still piss away a lot of time), but the result is my own company, a lot of success, etc.
7 replies →
Some of us are 9-5 types without kids. I strongly believe that the work yourself to death model isn't healthy, and in my experience doesn't lead to the best code long term either. I see it all the time from college hires though.
A generalization is a generalization. OP didn't write this, merely reposted it for the benefit for the rest.
Understood, and this was more of an open-ended rebuttal not directed at OP.
I couldn't agree more. That one phrase was the only thing that bugged me. As if having kids suddenly made me a lower quality coder. I wonder how many of the seasoned veterans that the author lamented losing had kids.
As people get older, they get more experience, which is good. They are also more likely to have a family. Is that supposed to be bad?
I think there are two complementary parts that Linux gets right here. One is as cited in the article, that you get personal glory for improvements you make to the kernel even if they are fairly small. The other part is that there is for Linux someone who will say "no" to a patch. Linus will certainly do it, and other trusted devs will do it too. Linus is perhaps even famous for telling people when they are wrong.
I've seen a number of open-source projects where you get all the personal glory for your additions but there is nobody who takes the responsibility to tell people "no". These projects, almost universally, turn into bloated messes over time. Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games.
> Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games.
I just listened to an episode of Roguelike Radio from the mid-30s (maybe the one with Red Rogue or Thomas Biskup) and Andrew / Keith (IIRC) go on and on about how all the "major" roguelikes are just accretions of features.
This rings very true to me; I grew up playing Nethack [0], and Slash (and then Slash'Em) just got silly with all the things they were adding. Light sabers ? In my roguelike ? Come on, fhgwgads. Nethack at least has a mysterious Dev Team who ostensibly are the guardians of quality... but they _did_ bring Sokoban back from Slash'EM into vanilla.
[0] since Nethack was my first roguelike, I didn't realize until lately just how crappy it is in many ways, most notably that it's very difficult to win before you've figured out all the sources of instadeath (whether you've figured them out by spoilers (written or source-diving) or through (really frickin hard-won) experience).
Try POWDER. It's made practically by one person, and it's very well designed (inspired by Dungeon Crawl Stone Soup). It also has hands down the best writing in any roguelike. God system in particular is unique, all 5 or so gods evaluate your behavior at the same time. Gods like different actions, and whatever you do you will piss off some gods and delight others. It requires some spoilers to understand, but is very fun once you do. POWDER's implementation of Xom is intriguing - he's mentally unstable and changes his preferences every 1000 turns, first he wants you to act like a warrior, second he gets annoyed when you strike a monster in melee. Or even kill anything personally when he impersonates Pax...
But I digress. POWDER feels very much a designed roguelike, not something that evolved.
Try brogue, it's a breath of fresh air.
1 reply →
The typical scenario you will find in any big corporation developing software.
I started to understand better how Microsoft works, after getting into the Fortune 500 enterprise world.
Many of the bad things geeks associate with Microsoft are actually present in any development unit of big clunky Fortune 500 companies.
What's wrong with 9-5? Can't you be passionate about what you work on, excel in you career, yet stick to dedicating ~50% of your time awake to your job?
It is an American thing. In the rest of the world we can happily do 9-5 without feeling bad for it.
Unfortunately that kind of thinking is no longer only an American thing. I see it more and more every day. Employers are pushing fresh college grads to work as many hours as possible for a low wage just so they can have a bigger profit. If they burn out, there are others to replace them.
It's amusing watching Americans boast about working past the norm 9-5 hours and calling the rest of the world lazy. My response, "What are you proud about? After working twice as hard, you have the same quality of life as friends in Canada, and statistically speaking you'll drop dead sooner because of stress."
That always wipes the smug grin of their faces.
2 replies →
Because you spend 9-noon and 1pm-3pm on meetings and you still need to code.
That sounds like an organizational problem to me if you spend 75% of your time at meetings everyday instead of coding (if coding is actually your job).
And 8hrs is probably too much time to really productively be working on a job at that.
We can't touch named pipes. Let's add %INTERNAL_NOTIFICATION_SYSTEM%! And let's make it inconsistent with virtually every other named NT primitive.
Linux does this motherfucking bullshit too. "Oh, systemd is piss slow? We'll bung d-bus straight into the kernel, right along side the over 9000 other IPC mechanisms. Everybody uses d-bus these days, it's an Essential System Component. What? Systemd is a crap idea to start with? People like you should be committed."
Considering linux actually hasn't done this yet, that seems like a weak argument. Are there better examples?
Update in the original article:
Besides: you guys have systemd, which if I'm going to treat it the same way I treated NTFS, is an all-devouring octopus monster about crawl out of the sea and eat Tokyo and spit it out as a giant binary logfile.
I keep coming here, writing a bunch of stuff, then removing it.
The author isn't far off. He's over the top on some things, but on the whole it's a good take on some of the reasons Microsoft culture is toxic.
My take:
1. The review system needs to be fixed. They have to stop losing good engineers.
2. They have to somehow get rid of a bunch of toxic people. Many of these are in management. It's hard.
3. They have to stop treating Windows as "the thing we put in all the other things".
Windows is a fine OS; it could be better, and the OP points out a lot of reasons why it isn't. But it's not a great fit for embedded systems, nor game consoles, nor anything where you're paying for the hardware.
But I keep coming back to the review system rewarding the wrong people, causing good people to leave. The underlying backstabbing and cutthroat culture needs to go; it's hurting the bottom line, and I'm surprised the board has been willing to let it happen.
Ah, so the symlink plan is true: https://gist.github.com/800407/3588729d57abb7731985c99c492a2...
> These junior developers also have a tendency to make improvements to the system by implementing brand-new features instead of improving old ones.
That's exactly the problem that's plaguing Google Chrome right now, although probably for a different reason, as many senior developers still seem to be on board. Google keeps adding new features at high pace and doesn't care what brakes in the process. The amount of unfixed (albeit mostly minor) bugs is huge.
Backward compatibility is an OS performance metric. Maybe sales is too. Microsoft has to think long and hard about any kernel change. In some irony, Microsoft doesn't own the Windows code, and any individual can own the Linux kernel - i.e. Windows lacks forks.
That Microsoft discourages individual junior developers from cowboying, is a point in their favor. Optimization for its own sake is not what benefits their users - real research does.
We just can't be fucked to implement C11 support
I thought Microsoft refusing to update their decades behind, ancient C compiler was just to piss me off and make life difficult cross-platform developers that need to work in C. Interesting to see this applies to their own employees too.
I believe the official policy is that C++ is the upgrade path from C, and C is therefore deprecated.
I think a lot of people that care about C support are already aware of Herb Sutter's stance on the matter. But knowing why he made his decision doesn't really change anything, its not like C++ never crossed their minds before. And classing it up by calling it official policy doesn't make the pill go down any easier.
Microsoft's position in the market is already big enough that they'll never really be proven wrong on C unless they relent. But they're also not large enough to kill it outright.
1 reply →
tl;dr : corporatism and careerism. It's the death of creativity and productivity. No large organization is immune to it. Never will be. (and yes, that includes Google... it's just much smaller and newer than Microsoft is right now)
michaelochurch has talked about this before, including the fundamental flaws behind it.
He's also a vindictive windbag who is extremely disgruntled about his shortcomings at Google.
2 replies →
I've even seen this mentality in startups. It does have some business rationale, provided you are thinking short term and focused only on near-term goals.
One of the reason businesses have trouble really innovating is that it's hard in a business to work on long-term things when markets are very short sighted. Only mega-corps, monopolies, and governments can usually do that... or hobbyists / lifestyle businesses who are more casual about hard business demands.
That being said, MS is surely cash-rich enough to think long term. So this doesn't apply as much here.
I've also found that of all things optimization almost gets you looked down upon in most teams -- even young ones. "Premature optimization is the root of all evil," and all that, which is usually misinterpreted as "optimization is naive and a waste of time." It's seen as indicative of an amateur or someone who isn't goal-focused. If you comment "optimized X" in a commit, you're likely to get mocked or reprimanded.
In reality, "premature optimization is the root of all evil" is advice given to new programmers so they don't waste time dinking around with micro-optimizations instead of thinking about algorithms, data structures, and higher order reasoning. (Or worse, muddying their code up to make it "fast.") Good optimization is actually a high-skill thing. It requires deep knowledge of internals, ability to really comprehend profiling, and precisely the kind of higher-order algorithmic reasoning you want in good developers. Most good optimizations are algorithmic improvements, not micro-optimizations. Even good micro-optimization requires deep knowledge-- like understanding how pipelines and branch prediction and caches work. To micro-optimize well you've got to understand soup-to-nuts everything that happens when your code is compiled and run.
Personally I think speed is really important. As a customer I know that slow sites, slow apps, and slow server code can be a reason for me to stop using a product. Even if the speed difference doesn't impact things much, a faster "smoother" piece of code will convey a sense of quality. Slow code that kerchunks around "feels" inferior, like I can see the awful mess it must be inside. It's sort of like how luxury car engines are expected to "purr."
An example: before I learned it and realized what an innovative paradigm shift it was, speed is what sold me on git. The first time I did a git merge on a huge project I was like "whoa, it's done already?" SVN would have been kerchunking forever. It wasn't that the speed mattered that much. It was that the speed communicated to me "this thing is the product of a very good programmer who took their craft very seriously as they wrote it." It told me to expect quality.
Another example: I tried Google Drive, but uninstalled it after a day. It used too much CPU. In this case it actually mattered -- on a laptop this shortens battery life and my battery life noticeably declined. This was a while ago, but I have not been motivated to try it again. The slowness told me "this was a quick hack, not a priority." I use DropBox because their client barely uses the CPU at all, even when I modify a lot of files. Google Drive gives me more storage, but I'm not content to sacrifice an hour of battery life for that.
(Side note: on mobile devices, CPU efficiency has a much more rigid cost function. Each cycle costs battery.)
Speed is a stealth attribute too. Customers will almost never bring it up in a survey or a focus group unless it impacts their business. So it never becomes a business priority.
Edit: relevant: http://ubiquity.acm.org/article.cfm?id=1513451
> In reality, "premature optimization is the root of all evil" is advice given to new programmers so they don't waste time dinking around with micro-optimizations instead of thinking about algorithms, data structures, and higher order reasoning. (Or worse, muddying their code up to make it "fast.")
It's also for experienced programmers who dink around with macro-optimizations. For example, designing an entire application to be serializable-multi-threaded-contract-based when there's only a handful of calls going through the system. Or creating an abstract-database-driven-xml-based UI framework to automate the creation of tabular data when you have under a dozen tables in the application.
premature optimization is the root of all evil is a really really important mindset, and I agree it doesn't mean to not optimize, and many developers seem to take it that way.
X+1 = How many transactions your business does today
Y = How many transactions your business needs to do in order to survive
Y/X = What the current application needs to scale to in order to simply survive. This is the number where people start receiving paychecks.
(Y/X)4 = How far the current application needs to scale in order to grow.
The goal should be to build an application that can just barely reach (Y/X)4 - this means building unit tests that test the application under a load of (Y/X)4 and optimizing for (Y/X)4
Spending time trying to reach (Y/X)20 or (Y/X)100 is what I'd call premature optimization.
Disclaimer: (Y/X)4 is no real point of data that I know of, just something I pulled out as an example, anyone who knows of actual metrics used please feel free to correct.
> > In reality, "premature optimization is the root of all evil" is advice given to new programmers
> It's also for experienced programmers who dink around with macro-optimizations.
The hilarious thing is that everyone seems to think everyone else's optimizations are premature.
Don't believe me? Just go ahead and ask someone an optimization question, like maybe on StackOverflow. Those who respond might not know you, but I'll be damned if their first reply isn't to tell you that you probably shouldn't even be optimizing in the first place, because obviously if you'd thought about it more carefully, you'd have realized it's premature.
3 replies →
The canonical version is Alan J. Perlis Epigram 21:
If you have a black box, then optimize the fuck out of it. The Windows kernel is not a black box.
6 replies →
"Or creating an abstract-database-driven-xml-based UI framework to automate the creation of tabular data when you have under a dozen tables in the application."
I'm not an experienced programmer, and I'd use that approach anyway, simply because it makes sense (is there any alternative that isn't worse?). What does that mean? (Oh, and I'd skip XML. I abhor violence.)
7 replies →
i admit to doing this... made a multithreaded simulation framework for use by my game team.... too bad the games they make with are so computationally simple as to make the sim framework a complete waste :(
The concept of "premature optimization" also has another connotation in product development: Don't waste too much time making that product or feature optimized until you are convinced you can actually sell it. It's not that optimization is bad, but optimization before market trial (premature) can result in you spending precious time working hard on the wrong thing.
Optimizing the right thing is good, but figure out what that thing is first.
In a publicly traded company, such as Microsoft, that doesn't take long. It is shareholder value. It is not Optimization "for its own sake" promoted in the article. I savvy developer made need to run business tests against their software.
8 replies →
Measurement can confirm or potentially at the limits replace the deep understanding you discuss.
Disclosure: I'm a new guy at Microsoft, not related to Kernel or Windows stuff.
So where does he back up the claim that NT kernel performance is, in fact, "behind other OS"?
Slightly off topic, but is there anyway we can actually confirm that "the SHA-1 hash of revision #102 of [redacted] is [redacted]."
EDIT: redacted information. Still, when that information we present, how would anyone be able to confirm it?
The article redacted that data on the request of the original poster; please remove it from your post.
I assume some other dev with source access at Microsoft could verify that fact, broadcast that the fact is true, and thereby stamp the OP as actually who he says he is.
From what I've heard (unofficially) the source is typically watermarked to identify the sources of any leaks. He may have given himself away with the SHA-1.
I doubt it. Not only would that involve generating a different watermark for each person with access to the code, but in doing so you also raise complications for source control. Additionally, it is almost impossible for developers not to notice if supposedly identical files are not identical, not to mention the fact that implementing the type of system without the conscious corporation of the developers seems impossible.
I had the same question but was too embarrassed to ask. Thanks for doing it for me. :)
Given the topic of the article I understand my question could be seen as OT but I'd like to dig more into the following:
> The NT kernel is still much better than Linux in some ways --- you guys be trippin' with your overcommit-by-default MM nonsense
What's wrong with the overcommit-by-default behaviour? Or how/why the NT kernel is supposed to do better?
I don't know what the author was thinking of specifically, but the OOM killer seems like a bit of a hack to me. I can think of situations where I'd rather have an allocation fail upfront predictably than be randomly killed by the system.
A lot of people seem to think you're screwed when you're out of memory and so think killing the process is acceptable, but I've worked on code bases where it's actually handled somewhat gracefully. (Although, on NT, if you run out of nonpaged pool you start to get weird, random I/O failures.)
but again, if it is just about the default behaviour which we debate, one can simply disable the overcommit
vm.overcommit_memory = 2
see https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
1 reply →
UTF-16 everywhere doesn't help either. M\0o\0r\0e\0 \0d\0a\0t\0a.\0
You do realize that NT is from 1989 and that UTF-8 is from 1992, right?
Wikipedia tells me that Windows NT was started in 1988 and shipped in 1993, while Linux's first release was in 1991, giving them approximately the same age. Furthermore, NT shipped with UCS-2 support, not UTF-16, as UTF-16 did not exist until 1996. UTF-16 was a migration that they completed in Windows 2000. UTF-8 was first presented in 1993 and is, therefore, older than UTF-16.
With all of these facts being true (I hope), today things are such that the Windows world mostly (but not completely -- they sometimes assume two-byte characters) correctly implements UTF-16, while the Linux world correctly implements UTF-8.
1 reply →
Yeah. Which makes it even more irritating.. MS is still off in the double-wide stix 11 years later and still promoting UTF-16LE in userland. From http://msdn.microsoft.com/en-us/library/dd374081%28VS.85%29....
"Unicode-enabled functions are described in Conventions for Function Prototypes. These functions use UTF-16 (wide character) encoding, which is the most common encoding of Unicode and the one used for native Unicode encoding on Windows operating systems. Each code value is 16 bits wide .. New Windows applications should use UTF-16 as their internal data representation."
3 replies →
There is a great quote in the post: "Incremental improvements just annoy people and are, at best, neutral for your career."
Working for several different Fortune 500 companies this has been my experience many times. I have worked on products where I made a large number of incremental improvements that never got incorporated into the released product. Taken on their own the improvements don't change things much, but in the aggregate they make a significantly better product. Why didn't most of my incremental improvements get incorporated into the product? Every change, no matter how minor, is perceived by management to be a highly risky proposition. Product managers won't get promoted because the product is 4% better, but they could potentially get fired if that same incremental improvement had an unforeseen negative consequence.
Once the short term sales mindset gets power, hit products are viewed as cows to be milked and not as things that need continued maintenance and improvement. As an engineer in such an enterprise you watch helplessly as your competitors slowly but steadily pass you, and then someday upper management wonders why the magic money machine broke down. Some large enterprises avoid this fate but they seem to be the exception.
The problems with the NT kernel are nothing new. When I suggested for Windows to use a BSD kernel to overcome its current limitations, I got lots of disagreement. Microsoft developers seem to be happy with the current state and have no vision of a totally new architecture.
https://news.ycombinator.com/item?id=2841934
People are praising BSD like it was all hot and new. OS X is built on a BSD which has it's roots in 60's and 70's OS design, just like the VMS roots of WinNT.
OS X didn't change the world by bringing some great new underlying architecture to the table. In fact, their kernel and filesystem are arguably getting long in the tooth. The value that OS X brought to the table was the fantastic Carbon and Cocoa development platforms. And they have continued to execute and iterate on these platforms, providing the "Core" series of APIs (CoreGraphics, CoreAnimation, CoreAudio, etc.) to make certain HW services more accessible.
There's very little cool stuff to be gained in the windows world by developing a new kernel from scratch. A quantum leap would not solve MS's problem. The problem is the platform. What's really dead and bloated is the Win32 subsystem. The kernel doesn't need major tweaking. In fact, the NT kernel was designed from the beginning such that it could easily run the old busted Win32 subsystem alongside a new subsystem without needing to resort to expensive virtualization.
Unfortunately, the way Microsoft is built today it have a fatal organizational flaw that prevents creating the next great Windows platform. The platform/dev tools team and the OS team are in completely different business groups within the company. The platform team develops the wonderful .NET platform for small/medium applications and server apps while the OS team keeps crudging along with Win32. Managed languages have their place, but they have yet to gain traction for any top shelf large-scale windows client application vendors (Adobe, even Microsoft Office itself, etc.) Major client application development still relies on unmanaged APIs, and IMHO the Windows unmanaged APIs are arguably the worst (viable) development platform available today.
What Windows needs is a new subsystem/development platform to break with Win32, providing simplified, extensible unmanaged application development, with modern easy-to-use abstractions for hardware services such as graphics, data, audio and networking. This is starting to come to fruition with WinRT, but the inertia in large scale apps is unbelievable.
The NT-based kernels were not as big a problem for Microsoft as CE-based OSs were. Nokia transitioned away from a Linux-based smartphone OS to Windows Phone 7, based on Windows Embedded CE 6. It is clear that Windows Phone 7 was uncompetitive with other smartphone OSs. That put Nokia in limbo for a year when they could least afford it.
Now the problem is not so much the kernel, but all the cruft a Windows-based OS drags along with it. The most visible symptom is the amount of storage used up by stuff that adds little or no readily apparent value to Windows RT.
Windows OSs are heavy compared to Android and other Linux-based OSs. That means inexpensive tablets and phones won't run Windows, and Windows is shut out of mobile device markets in India, Africa, and other developing areas. These markets are how Yulong (who??!!) became a top 10 mobile OEM.
As an outsider, it looks like MSFT does not grasp kernel performance as a super important feature of Windows. They add so much bells & whistles over the UI, but the inside sucks release after release
As someone else posted in this same thread: is modern Windows substantially slower than the OSes it competes with (in anything other than a few uncommon niches)? If it's more than a few percentage points slower for anything but rare scenarios, I'd be surprised.
The description sounds accurate compared to what I heard when I was in contact with Microsoft insiders. That was about 15 years ago. What's surprising to me is that it is still this way.
A lot of the justification for being exceedingly conservative about performance improvements was based on the difficulty of doing testing that is sufficient to leave no doubt that changes don't cause regressions nor any side effects bearing on compatibility.
I'm surprised that hasn't changed, or that the value equation about possibly breaking customer software hasn't changed. By now this approach must be a very entrenched culture.
I think the main performance problem with NT is its scheduler. Compared to unix-type schedulers, it doesn't give very good interactive performance when there are cpu-bound processes. There have been a lot of hacks over the years (like throwing cpu-starved processes a bone when they haven't had any action in 4 seconds), but unix/linux just has a better designed scheduler that gives better performance in most situations.
I suspect another aspect is for linux developers to be less worried about keeping the API/ABI stable.
The Linux kernel developers take backwards compatibility very seriously. Here is a tame excerpt from Linux on the subject: "Seriously. Binary compatibility is so important that I do not want to have anything to do with kernel developers who don't understand that importance. If you continue to pooh-pooh the issue, you only show yourself to be unreliable. Don't do it." [1]
[1]https://lkml.org/lkml/2012/3/8/495
Why is it that I can run my Radeon x1900 under Windows 7/8 with Windows Vista drivers? Runs StarCraft2 and a bunch of newer FPS games just fine.
Linux on the other hand is a disaster. The kernel devs are so determined to break binary compatibility, I haven't been able to run with ATI's proprietary binary drivers for years. While AMD was a good open source citizen and released the specs, the open source drivers for my card are useless for anything other than 2D.
2 replies →
"The rot has already set in"
Great words for concluding a video game intro
At some point Microsoft might want to consider expanding its kernel team into Silicon Valley.
I support conservative engineering for file systems.
NTFS is highly reliable and Microsoft wants to build on that success with ReFS. In years of working with NTFS I've never had a serious filesystem wreck (although I did have one that was never quite the same after I made a few hundred million small files)
The NTFS story isn't that different from the ext(2|3|4) story on Linux. ext-N filesystems are boring and not particularly fast, but they're reliable. I've tried other filesystems on Linux but have often had wrecks or noticed data corruption.
I like boring filesystems.
I'm a recent college graduate starting on a Windows Core team in the next few months.
Whee. Talk about a downer to start my day.
Look at it this way: you're lined up to work in a company with far more capability and potential than almost every other western organisation. When you're compared to the likes of Google and Apple as examples of where you might be lacking, you're doing good.
Besides, even if everything is as bad as they say and you absolutely cannot work in that environment, interning is the perfect time to discover that without as significant commitments. It will also prove to be a valuable lesson that you can then take to other companies.
Chin up, I'd love to work even at Microsoft.
"our good people keep retiring or moving to other large technology companies, and there are few new people achieving the level of technical virtuosity needed to replace the people who leave."
in the post PC era Kernels are not as hot as they were in 2000
Isn't Windows NT used only in large corporations that are afraid to upgrade their servers?
The Windows NT _Kernel_ is used in Windows NT, 2000, XP, Vista, 7, 8, etc. (Versus the DOS kernel used in DOS, 95, 98, and Me)
Yea, that is another pet peeve of mine. Windows 2000 is technically NT 5.0 (and was called that during the beta). XP is NT 5.1 etc...
Not to mention Windows Phone 8.
Is performance behind other OS'? Serious question, because empirically I'm not aware of any particular performance lag.
The problems with the NT kernel are nothing new. When I suggested for Windows to use a BSD kernel to overcome its current limitations, I got lots of disagreement. Microsoft developers seem to be happy with the current state and have no vision of a totally new architecture.
https://news.ycombinator.com/item?id=2841934
Pig say "what? I aint no chicken!"
Windows is dying just like DOS.
Azure is the new Windows.
I don't know what this is even supposed to mean.
Sad post IMO, which will unfortunately and unduly tarnish Microsoft's reputation. I would suppose it was written by a developer who has just become senior enough to see some of what's going on, but has not yet realized it's everywhere like that. One day, he'll understand that those challenges too need to be managed.
Growing pains, all around.