What Happened to Software Development?
I began writing this in the third person as an objective description of the changes I’ve witnessed in my three decades of writing software, and I got bored just proofreading it. I’m instead writing something of a personal odyssey, my own experience, a framing that adds a little texture and leads to how bizarre and wrong I find today’s world of software development.
To Younger Developers
Many who are too young to have ever seen the halcyon days of software development will not understand my reminiscences of the formerly sane environment we worked in, how vital it was that we were allowed to work uninterrupted almost all the time. Just as my teenage students in Vietnam have barely heard of the Beatles and have no idea why we regard them as so important in music, you in your 20s and 30s have never seen a software office where you were not in constant communication and never able to focus on your work for very long. You cannot possibly know what it was like. You should try to find out.
This is my advocacy; this is the awareness I want to bring back. I don’t care about Agile or Scrum or Extreme Programming, I regard these as silly fads and since they interfere with concentration they cannot help. It is no exaggeration to note that Microsoft’s early greatness was the direct result of a corporate culture built around enabling unbroken concentration and the disappearance of that greatness is directly attributable to abandonment of that awareness.
I have a reference to my short article just below but if you would like a scholarly explanation with the results of research you should read this:
I started getting paid to write software in 1988. I was at Microsoft a year later and saw the last year of single-occupancy offices and minimal interruptions. It was great back then; we were treated like princes and we got more work done than anyone before or since. The company built a culture around recognition of Flow, enabling entering and maintaining that condition because that was how we did out best work.
Their IPO had been shortly before I arrived but the real change came with the release of Windows 3.0 in May 1990. The environment changed overnight; suddenly I shared an office with a smoker with bad BO who talked loudly on the phone all day. We started having more meetings.
I worked at Microsoft almost exactly half of 1989–2009 as both full-time employee and contractor and each time it was worse, culminating with the Windows Vista (Longhorn) project.
The Fall from Greatness
I first joined MS in 1989 as FTE. I was one year into being a programmer, I was in QA for OS/2 3.0, Cruiser. Five…
You’ve never seen people more exhausted or stressed, we were almost serfs. We worked 70 or more hours a week and we were lucky to get 4–6 hours of actual work done in that time; all the rest was wrestling with the checkin system, with its unfinished and barely-working quality gates as obstacles to our work.
By 2009 things were in disarray; the love of quality had completely given way to a mechanical process of checking boxes and several times I landed in hot water for being too thorough. That never would have happened in 1989; rigor was the highest virtue. My threat model for Windows Media Player DRM was 20 pages of vulnerabilities and mitigations; they wanted a page or two since every vulnerability had to be reviewed and answered.
Excellence was gone, dead and buried. When my manager told me in late 2008 to write doggerel code outside my application so they could run unit tests on that (and check the “has unit tests” box for the project) I started looking for an exit; when he told me about a new thing called “Test-Driven Development” I decided to update my CV and plan to get out of there. It’s a pity I didn’t do that right away, worse was to follow. When they had me do pair programming I resigned my contract in anger the next day.
Ten Years Ago: Agile
I took a job at Real Networks in downtown Seattle; driving is a big problem in Seattle, there are a lot of people whose mission in life is keeping other drivers well under the speed limit. But since I left at 9:30, after rush hour, my commute was 30 minutes and wasn’t so bad.
Then the group I was contracting in took up a new thing called “agile.” All this meant to me was that I had to attend a “morning standup,” an 8:30 meeting requiring me to get to work 90 minutes earlier. This meant I had to drive in rush hour traffic, turning my 30 minute commute to a 90 minute slog, arriving barely in time and in a weary foul mood. I asked if it could be scheduled later. No, don’t you know it’s a morning standup? (We didn’t stand up).
This meeting was supremely ridiculous, going through the motions. We would go around the room, each developer intoning that he was working on the same thing he’d been working on yesterday, or, occasionally, working on something new with nothing yet to report.
The PMs were worse, putting on a perky, chirpy and cheerful show of sounding “engaged,” when in reality I knew they spent most of their day playing games on Facebook. I’d heard the word “stories” a few times before I realized it wasn’t a reference to one of the games we sold. Excuse me, what do you mean by “stories?”
It turned out to be a new name for what we had heretofore called user scenarios, usage cases, you know, actual descriptive terms. The more I learned about agile the more renaming and rebranding I ran into and so far I still haven’t seen much added value. More meetings, though.
I protested “stories.” It felt infantilizing, like Safeway cashiers forced to wear silly costumes to promote some sale or acknowledge some holiday. As with the meeting time, I was frostily told that “stories” was part of agile and that we were going to follow the script to the letter. And, no, even though the “scrum” was nothing more than a status update we couldn’t do it in email as I had done for the last 20 years.
For this I spent an extra (unpaid) hour driving to work. O Brave New World.
Twice in 2010 I came out to Vietnam to see my new house, first under construction and then living in it for three weeks. My contract at Real had been extended even though half the company was laid off while I was there but it finally ran out, ending most pleasant job I’d had in a long time. I got along with everyone, I had some major achievements, the only jerk there was someone I didn’t work with and who went in one of the layoffs.
I had planned to move around 2019 when I reached retirement age. But the thought of going through dreary whiteboard interviews and a series of high-stress jobs (talking about “stories”) made me sick and I had so enjoyed living in my beautiful new house that I decided to pack up and go, and left the USA on 11/15/2010, intending to retire at 56, play my guitars, read my physics library, live my life in very strange language and relax.
I did learn to speak Vietnamese; otherwise I was bored out of my mind. I wasn’t meant for idleness. I didn’t do squat.
A friend suggested learning to write for the iPhone and iPad; the tools were free, and I missed programming. I bought a MacBook, learned iOS, Objective C, and XCode and soon had an app for sale. I was back in the game.
From 2011 to 2016 I wrote iOS and MacOS applications, first for myself and then for clients. It was okay but I wanted more money so I started going through freelancing agencies. In 2017 I got a job doing servers for a company in California; I learned C#, Entity Framework, ASP.NET and when the friend who had recommended me drifted away from the job I took over servers and database. This lasted 30 months. It was a great gig and got me some current skills. I love servers and databases.
All this time I was working solo. I was part of a team but the divisions were very high; a browser developer in Sydney and me in Vietnam. We had to collaborate on the REST API but we both worked independently.
The Perplexing Present
When that ended last August I figured finding more remote work would be easy, that my independence and readiness to take broad responsibility would both be assets, as they always had been.
I found a software industry changed beyond recognition.
Jargon and Fads
“Stories” was just the beginning. At my previous job we had two development leads who spoke in serial jargon. In addition to recognizable phrases like “industry standard,” which was supposed to be a supporting argument, there were others like “technical debt” that turned to be a non-blaming way of saying “unfinished work.” “Branch hygiene” translated to Doing Nothing about accumulating Git branches
The industry is now laden with jargon. “Agile” is everywhere, meaning something different for everyone who mentions it, same for “refactoring” which nearly as I can tell is a synonym for “edit.” A “sprint” is a small milestone with an overtone of overwork, which is definitely nothing new.
Well, there have been user scenarios, unfinished work, and editing as long as I’ve been in the industry and the old words worked just fine. Small milestones are not revolutionary. Overwork I don’t need to say anything about.
I think if I had to sit through a “sprint retrospective” (“let’s spend four hours discussing what we just learned about teamwork”) I would go stark raving mad.
Piling on additional meetings, sure as hell nothing new about that.
Independence As Conceit
This is one of the creepier new ideas; people such as myself who can manage large projects alone aren’t stellar developers, we’re conceited tyrants and prima donnas headed for demotion or termination. It’s all about teams, working together, playing foosball at work, putting feet up on the team table, holding alcohol-fueled team morale sessions, and doing mob- and pair-programming.
This is Hell’s innermost circle.
Oddly enough, job postings still seem to favor developers who can work independently, with minimal supervision and high initiative. Maybe someone needs to tell them that these are bad people and really they want to hire teams who work like Siamese twins.
The End of Flow
Open offices like the titular graphic; collaborative programming means sitting in excruciatingly uncomfortable proximity with someone in constant communication in what is supposed to be a continuous code review but which makes concentration entirely impossible. The constant din of conversation, no door to close for silence and focus, wearing headphones means you’re not a team player.
It was Flow that created greatness in the past, taking it away sets mediocrity as the highest attainable bar.
Development is Secondary to Testing
This final entry is probably the weirdest change in software development.
Admittedly, in the past we didn’t take testing seriously. There was a standing joke at Microsoft that nobody should use an even-numbered software version because it depended on customers reporting bugs; don’t use version 2.0 because version 2.1 will fix all the bugs reported by customers, at least the ones triaged as worth fixing.
I didn’t laugh.
I think we are now seeing an overreaction to this, spurred by the faintly ridiculous approach called Test-Driven Development.
Test-Driven Development is Fundamentally Wrong
It was late 2008 and I was writing an extension for the Windows 7 taskbar. My work was essentially finished, the final…
I’ve read in many discussions that
- there is nothing more important in software than the unit tests; they are more vital than the deliverable itself
- documents are obsolete; the unit tests are the design; they are where the API are defined
- writing tests to a completed design is inferior to writing tests to a pre-implementation guess
- anything short of 100% test coverage is dereliction of duty, 100% coverage is a point of honor
- developers are solely responsible for testing their own work, we don’t need blackbox testers anymore, we don’t need a second pair of eyes
I don’t think I need point out the fanaticism of these attitudes.
The last one is self-evidently absurd to anyone with any experience; every one of us has blind spots, cases we overlook, and we will overlook the same cases writing tests that we will overlook writing code. This is elementary, not subtle.
Never Onsite Again
I love writing software. I love solving problems and developing functionality. I’ve loved it since I wrote a GW-BASIC program to generate lists of prime numbers in 1984, I’ve loved it since I wrote a program in COMPASS Assembly language in 1967. It was a hobby that became a profession.
But it’s crazy now. I could not work in an open office, I wouldn’t last a week listening to jargon and dealing with the compulsive conformity and the recurring meetings and the derisive sneers at people who work alone. I hope the industry comes back to something like what it was in 1990 with mental focus valued and encouraged.
I loved my server development work, and I hope to find a niche where I can do it again. But it’s not going to be in any place that does pair programming and uses invented neologisms for old things.
I’m switching over to technical writing, I have a new job doing that for a company in Japan from the solitude and solace of my home in Vietnam. Meanwhile I will learn new skills that are in demand for remote work and hope to find more server and database development in environments I can stand.
Environments that don’t seem crazy.