Wide Awake Developers

Main

The Future of Software Development

I've been asked to sit on a panel regarding the future of software development. This is always risky and makes me nervous, for two reasons. First, prediction is a notoriously low success-rate activity. Second, the people you always see making predictions like this are usually well past their "use by" date. Nevertheless, here are a collection of barely-related thoughts I have on that subject.

  • Two obvious trends are cloud computing and mobile access. They are complementary. As the number of people and devices on the net increases, our ability to shape traffic on the demand side gets worse. Spikes in demand will happen faster and reach higher levels over time. Mobile devices exacerbate the demand side problems by greatly increasing both the number of people on the net and the fraction of their time they are able to access it.

  • Large traffic volumes both create and demand large data. Our tools for processing tera- and petabyte datasets will improve dramatically. Map/Reduce computing (a la Hadoop) has created attention and excitement in this space, but it is ultimately just one tool among many. We need better languages to help us think and express large data problems. In particular, we need a language that makes big data processing accessible to people with little background in statistics or algorithms.

  • Speaking of languages, many of the problems we face today cannot be solved inside a single language or application. The behavior of a web site today cannot be adequately explained or reasoned about just by examining the application code. Instead, a site picks up attributes of behavior from a multitude of sources: application code, web server configuration, edge caching servers, data grid servers, offline or asynchronous processing, machine learning elements, active network devices (such as application firewalls), and data stores. "Programming" as we would describe it today--coding application behavior in a request handler--defines a diminishing portion of the behavior. We lack tools or languages to express and reason about these distributed, extended, fragmented systems. Consequently, it is difficult to predict the functionality, performance, capacity, scalability, and availability of these systems.

  • Some of this will be mitigated naturally as application-specific functions disappear into tools and frameworks. Companies innovating at the leading edge of scalability today are doing things in application-specific behavior to compensate for deficiencies in tools and platforms. For example, caching servers could arguably disappear into storage engines and no-one would complain. In other words, don't count the database vendors out yet. You'll see key-value stores and in-memory data grid features popping up in relational databases any day now.

  • In general, it appears that Objects will diminish as a programming paradigm. Object-oriented programming will still exist... I'm not claiming "the death of objects" or something silly like that. However, OO will become just one more paradigm among several, rather than the dominant paradigm it has been for the last 15 years. "Object oriented" will no longer be synonymous with "good".

  • Some people have talked about "polyglot programming". I think this is a red herring. Polylgot is a reality, but it should not be a goal. That is, programmers should know many languages and paradigms, but deliberately mixing languages in a single application should be avoided. What I think we will find instead is mixing of paradigms, supported by a single primary language, with adjunct languages used only as needed for specialized functions. For example, an application written in Scala may mix OO, functional, and actor-based concepts, and it may have portions of behavior expressed in SQL and Javascript. Nevertheless, it will still primarily be a Scala application. The fact that Groovy, Scala, Clojure, and Java all run on Java Virtual Machine shouldn't mislead us into thinking that they are interchangeable... or even interoperable!

  • Regarding Java. I fear that Java will have to be abandoned to the "Enterprise Development" world. It will be relegated to the hands of cut-rate business coders bashing out their gray business applications for $30 / hour. We've passed the tipping point on this one. We used to joke that Java would be the next COBOL, but that doesn't seem as funny now that it's true. Java will continue to exist. Millions of lines of it will be written each year. It won't be the driver of innovation, though. As individual programmers, I'd recommend that you learn another language immediately and differentiate yourself from the hordes of low-skill, low-rent outsource coders that will service the mainstream Java consumer.

  • Where will innovation come from? Although some of the blush seems to be coming off Ruby, the reduction in hype has mainly allowed Ruby and Ruby on Rails developers to knuckle down and produce. That community continues to drive tremendous innovation. Many of the interesting developments here relate to process. Ruby developers have given us fantastic tools like Gems and Capistrano, that let small teams outperform and outproduce groups four times their size.

  • To my great surprise, data storage has become a hotbed of innovation in the last few years. Some of this is driven by the high-scalability fetishists, which is probably the wrong reason for 98% of companies and teams. However, innovations around column stores, graph databases, and key-value stores offer developers new tools to reduce the impedance mismatch between their data storage and their programming language. We spent twenty years trying to squeeze objects into relational databases. Aside from the object databases, which were an early casualty of Oracle's ascension, we mostly focused on changing the application code through framework after framework and ORM after ORM. It's refreshing to see storage models that are easier to use and easier to modify.

  • This will also cause another flurry of "reactive innovation" from the database vendors, just as we saw with "Universal Databases" in the mid-90s. The big players here--Microsoft and Oracle--won't let some schemaless little upstarts erode their market share. More significantly, they aren't about to let their flagship products--and the ones which give them beachheads inside every major corporation--get intermediated by some open-source frameworks banged up by the social network giants. Look for big moves by these vendors into high scalability, agile storage, and eventual consistency storage.

JAOO Australia in 1 Month

Speaking at JAOO The Australian JAOO conferences are now just one month away. I've wanted to get to Australia for at least ten years now, so I am thrilled to finally get there.

I'll be delivering a tutorial on production ready software in both the Brisbane and Sydney conferences. This tutorial was a hit at QCon London, where I first delivered it. The Australian version will be further improved.

During the main conference, I'll be delivering a two-part talk on common failure modes) of distributed systems break and how to recover) from such breakage. These talks apply whether you're building web facing systems or internal shared services/SOA projects.

OTUG Tonight

This evening, I'm speaking at OTUG. The topic is "Clouds, Grids, and Fog".

There's no denying that "cloud" has become a huge buzzword. It's a crossover trend, too. It's not just the CIO who is interested in cloud computing. It's the CFO and the CMO, too. (Not to mention the CSO, if there is one.)  Underneath the buzz, though, there is something real and valuable.

I will talk about the driving trends that are leading us toward cloud computing and how it differs from grids and software-as-a-service. I'll also talk at length about the architectural implications and effects of running your software on a cloud.

If you live in the Twin Cities, I hope to see you there.

Arrival at JAOO

Considering that it's 7:30 AM local time---where "local" means Aarhus, Denmark---and I'm awake and online, it looks like I've successfully reset my internal clock.  Of course, my approach consisted of staying awake for 28 hours continuously then having three excellent beers with dinner.  There are probably easier ways, and there may be repercussions later.

I've always heard good things about JAOO, so it was an honor and a delight to be invited. So far, just hanging around the hotel has been interesting. Waiting to check in yesterday evening, I encountered Richard Gabriel and one of the guys who designed Windows PowerShell. (He still calls it Monad, which I think was a much better name than "PowerShell".  Also, I wish I'd gotten his name, but I was a too distracted by the problem with my reservation.)

After dinner, I started chatting with some ThoughtWorkers over a game of ZombieFluxx. Two observations: first, ZombieFluxx is the kind of game that only a computer programmer or a lawyer could love. The deck of cards includes many cards that change the rules of the game itself. Gameplay changes from turn to turn based on the current state of the rule cards showing. There's even a card that requires you to groan like the undead whenever you turn over a new "zombie" card. Very meta.  Second, it seems that TW people make up half of every conference I go to. They must have a fantastic training budget, because they are disproportionately represented relative to their much larger competitors like Accenture, Deloitte, and that crowd. Woe to the conference industry if ThoughtWorks falls on hard times.

My primary goal for today was to get over jetlag. Having accomplished that before 8 AM, I'll now see about straightening out my hotel situation. It's hard to think much about software when you may not have a roof over your head come nightfall.

Update: Got my hotel issues resolved. Now at a thoroughly modern, thoroughly Danish hotel called the "Best Western Oasia". Funny, but I always think of "Best Western" as the cruddy, mildewed cheap hotels off the Interstate in places like west Texas and Birmingham, Alabama. This hotel may cause me to reevaluate that image! It's nice, in a kind of "living inside Ikea" way.

(And, yes, I know Ikea is Swedish, not Danish. It's the bare wood, spare furnishings, and black lacquer I'm talking about.)

Agile IT! Experience

On June 26-28, 2008, I'll be speaking at the inagural Agile IT! Experience symposium in Reston, VA. Agile ITX is about consistently delivering better software. It's for development teams and management, working and learning together.

It's a production of the No Fluff, Just Stuff symposium series.  Like all NFJS events, attendance is capped, so be sure to register early.

From the announcement email:

The central theme of the Agile ITX conference (www.agileitx.com) is to help your development team/management consistently deliver better software. We'll focus on the entire software development life cycle, from requirements management to test automation to software process. You'll learn how to Develop in Iterations, Collborate with Customers, and Respond to Change. Software is a difficult field with high rates of failure. Our world-class speakers will help you implement best practices, deal with persistent problems, and recognize opportunities to improve your existing practices.

Dates: June 26-28, 2008

Location: Sheraton Reston

Attendance: Developers/ Technical Management

Sessions at Agile ITX will cover topics such as:

  • Continuous Integration (CI)
  • Test Driven Development (TDD)
  • Testing Strategies, Team Building
  • Agile Architecture
  • Dependency Management
  • Code Metrics & Analysis
  • Acceleration & Automation
  • Code Quality

Agile ITX speakers are successful leaders, authors, mentors, and trainers who have helped thousands of developers create better software. You will have the opportunity to hear and interact with:

Jared Richardson - co-author of Ship It!
Michael Nygard - author of Release It!
Johanna Rothman - author of Manage It!
Esther Derby - co-author of Behind Closed Doors: Secrets of Great Management
Venkat Subramaniam - co-author of Practices of an Agile Developer
David Hussman - Agility Instructor/Mentor
Andrew Glover - co-author of Continuous Integration
J.B. Rainsberger - author of JUnit Recipes
Neal Ford - Application Architect at ThoughtWorks
Kirk Knoernshild - contributor to The Agile Journal
Chris D'Agostino - CEO of Near Infinity
David Bock - Principal Consultant with CodeSherpas
Mark Johnson - Director of Consulting at CGI
Ryan Shriver - Managing Consultant with Dominion Digital
John Carnell - IT Architect at Thrivent Financial
Scott Davis - Testing Expert

Suggestions for a 90-minute app

Some of you know my obsession with Lean, Agile, and ToC.  Ideas are everywhere.  Idea is nothing. Execution is everything.

In that vein, one of my No Fluff, Just Stuff talks is called "The 90 Minute Startup".  In it, I build a real, live dotcom site during the session. You can't get a much shorter time-to-market than 90 minutes, and I really like that.

In case you're curious, I do it through the use of Amazon's EC2 and S3 services. 

The app I've used for the past couple of sessions is a quick and dirty GWT app that implements a Net Promoter Score survey about the show itself. It has a little bit of AJAX-y stuff to it, since GWT makes that really, really simple. On the other hand, it's not all that exciting as an application. It certainly doesn't make anyone sit up and go "Wow!"

So, anyone want to offer up a suggestion for a "Wow!" app they'd like to see built and deployed in 90 minutes or less?  Since this is for a talk, it should be about the size of one user story. I doubt I'll be taking live requests from the audience during the show, but I'm happy to take suggestions here in the comments.

(Please note: thanks to the pervasive evil of blog comment spam, I moderate all comments here. If you want to make a suggestion, but don't want it published, just make a note of that in the comment.) 


Conference: "Velocity"

O'Reilly has announced an upcoming conference called Velocity.

From the announcement:

Web companies, big and small, face many of the same challenges: sites must be faster, infrastructure needs to scale, and everything must be available to customers at all times, no matter what. Velocity is the place to obtain the crucial skills and knowledge to build successful web sites that are fast, scalable, resilient, and highly available.

Unfortunately, there are few opportunities to learn from peers, exchange ideas with experts, and share best practices and lessons learned.

Velocity is changing that by providing the best information on building and operating web sites that are fast, reliable, and always up. We're bringing together people from around the world who are doing the best performance work, to improve the experience of web users worldwide. Pages will be faster. Sites will have higher up-time. Companies will achieve more with less. The next cool startup will be able to more quickly scale to serve a larger audience, globally. Velocity is the key for crossing over from cool Web 2.0 features to sustainable web sites.

That statement could have been the preface to my book, so I'll be submitting several proposals for talks.

Pragmatic Podcast

Has anyone ever been happy to listen to their own voice? Probably not.

The Pragmatic Podcast is up and running on the redesigned Pragmatic Programmers site. In the first episode, Daniel Steinberg interviews me about the book.

Also available on iTunes.

Heads down

I've been quiet lately for a couple of reasons.

First, I'm thrilled to say that I'm joining the No Fluff, Just Stuff stable of speakers.  It's an honor and a pleasure to be invited to keep such company.  The flip side is, I'm spending a lot of my free time polishing up my inventory of presentations.  More frankly, I'm rebuilding them all with Keynote.  (Brief aside, I'm coming to love Keynote.  It has some flaws and annoyances, but the result is worth it!)

I'll debut the first of these new presentations at OTUG on May 15th.  I'll be speaking about "Design for Operations".  The talk will be about 70% from the last part of Release It, and about 30% original content.  OTUG will be giving away a couple of copies of my book, but you have to be there to win!

Finally, I'm working on an article about performance and capacity management.  Most capacity planning work is done entirely within Operations, without much involvement from Development.  At the same time, most developers don't have a visceral appreciation for how dramatically the application's efficiency can affect the system's overall profitability.

This article will show the relationship between application response time, system capacity, and financial success.  I'm hoping to include a simulator app for download that you can use to play with different scenarios to see what a dramatic difference 100ms can make. 

I forgot to mention that I will be speaking at OTUG

I forgot to mention that I will be speaking at OTUG on April 19th! I will be speaking on "Living With Systems in Production: Avoiding Heartbreak in Long-Term Relationships With Your Code"

From the summary of the talk:

Everything changes after Release 1.0. One batch of consultants leave, key developers jockey to get themselves reassigned, and the free-wheeling development environment is replaced by the painful rigor of operations. Or, at least, it should be. Systems in production require a different kind of care and feeding. If you have to live with a system in production, your quality of life is largely determined by the things you put in place before Release 1.0. This talk covers the topics that will give you God-like powers over your production systems. If you are an architect or developer who has ever put a system in production--or expects to put a system into production--then this talk is for you.

Much of this will be derived from my experiences at Totality and Best Buy. Spending time in operations gave me a great education about building systems to run, instead of building them to pass QA.

Technorati Tags: operations, OTUG, speaking