funkatron.com

This is going to make two blog posts in a row that are “off the cuff” for me. This one particularly so, because it involves a lot of emotion, and not a lot of certainty. There are some arguments to be made, but most of it is about how I feel. That’s something that’s hard to justify logically; it just is.

I’ve been frustrated with Twitter regarding how it relates to its developer community for several months now. My perspective might be different from some folks, I think, because I’ve been a part of that community for a long time. The Twitter Dev Talk Google Group shows me as joining on March 15, 2007. I remember being proud that I was asked to be a moderator for the group; it felt good to be both friends with some folks at Twitter, and a “Friend of Twitter” as someone from the company referred to me and a few others once.

It felt good because Twitter really seemed to make an effort to be open and transparent with their development community. We were part of what helped them get off the ground — not the biggest part by any means, but a part. At the time, the vibe I got from Twitter was that they were a group of people trying to build Cool Stuff. I liked that because I like building Cool Stuff too, for the sake of building it. It felt like it was being built by guys who had the same passion for making things that I do. I would hear that they really liked what I was doing in creating an open source, multi-platform client for Twitter; that it was important to have options like that.

Over the years I often found myself defending Twitter’s API team when folks would air their own frustrations. Some of those frustrations were valid, and some of them were based on really unrealistic expectations. I often said that it was foolish to build a business on top of Twitter, because (except in very rare circumstances) there was no guarantee of any service level of API feature permanence. Things were often pretty spotty back then in terms of uptime and performance; that frustrated a lot of people. But I had their back, ya know? I was a Friend.

Things grew, and things changed. That’s not really surprising at all. I’m happily naive of most things related to VC investment, but I suppose to keep up with all the traffic Twitter got as it became more popular, they needed money to pay for services and hire people. If you aren’t making any money, you need to ask people to give it to you. And now you have to worry about what they want, too.

In the past year or so, it felt like I was seeing a shift in how Twitter related to its developers. People move around in a big company, and that was surely part of it – I didn’t know all the New Dudes. And I had really withdrawn from the dev mailing list, because I was tired of dealing with spam and talking about uptime and downtime and social graphs and pagination limits.

But it got weirder when they announced Chirp, their “official developer conference.” Doing this event made sense, but I felt like some of their choices about who was speaking didn’t. I didn’t see guys like Cameron Kaiser, Abraham Williams, Marco Kaiser, JazzyChad, or Damon Cortesi listed as speakers. Instead, I saw execs, pundits, and VC investors.

Now I’ve been to a lot of developer conferences, both as an attendee and a speaker. And I’ve never seen a speaker lineup so devoid of actual developers. Anyone who knows me, knows that I generally believe pundits and VC investors (and execs to some extent) to be moronic at best, and pure evil at worst — corrupting the purity and passion for Making Cool Stuff.

Then there was how some stuff was handled. When Chirp was first announced in late January, before the speaker lineup was finalized, I inquired about speaking there myself. A friend suggested I talk about my experiences as an open source dev, and this seemed like a pretty darn good idea – I wanted to talk about what I’d learned, what motivates us as developers, and how we define success. So I made an official inquiry, and was told by a Twitter employee that they’d look into it and get back to me soon. I didn’t hear anything for a couple weeks, so I sent a follow-up, and was told that things were filling up on the main day (well, that’s why I asked early!) but that there might be an unconference opportunity. There’d be a public announcement about it in March.

Well, the conference is on April 15th. Not knowing until March what might happen is just not enough time to plan. And to buy a $450 ticket, and then hotel in SF, and then airfare; I just couldn’t afford that. Of course, it turns out that cheaper, hackday-only tickets would be made available, but these weren’t released until March 30, 15 days before the conference¹. Far too short notice, and by this time the speaker lineup described above was set, which indicated to me that this wasn’t a conference for developers like me.

So in this, I saw a pretty big chasm between where I was, and where Twitter was. Before then I hadn’t quite realized what it was like, but this kinda dumped on me like a ton of bricks. Getting a bit of the runaround hurt my ego, for sure, and that’s part of it. I don’t really think that’s a good reason to get upset, but I did feel like I had a longstanding relationship with Twitter, and through this, and the way Chirp was being presented, I didn’t feel like this was the same organization I’d interacted with for the past three years.

And, well, it wasn’t. They used to have 6 employees, and now they’re over 140. Duh.

So with all that, then we have Twitter dropping an enormous bombshell yesterday on their dev community, by creating official clients for both BlackBerry and iPhone. Twitter had done a few things before to add services that had previously been handled by third parties, but in general they’d had a very hands-off approach with the dev community, sticking with their core features and letting developers handle stuff like url shortening, photo uploading, and — most significantly — all non-web clients. Twitter has enjoyed a very diverse, very organic community of applications and services. Client apps that serve as primary user tools for interacting with Twitter are a very large part of that community. Twitter’s moves in that space have suddenly and dramatically changed the playing field for all of them.²

There was no warning, and there were no apologies. No statements made to developers that Twitter knew this was going to impact a lot of them; no attempt to unruffle feathers or explain how this was what Twitter needed to do as a company. We did get a couple notes on the dev list, but they could be summed up as follows: “it happened, now find other things to do.” That’s what angered me more than anything last night.

This really isn’t about my own client, Spaz. Spaz is not popular, and I doubt even having a directly-competing Official Client on desktop or webOS would impact its numbers – folks choose Spaz because they like it, and if they don’t that’s fine with me. Spaz exists as an alternative to closed source, commercial apps, and it’s even MORE important now that it exists, I think.

I guess it’s about respect. It’s about a relationship I had, and I think many of us had, with Twitter, that doesn’t seem to be there anymore. And that’s frustrating, and disappointing. But mostly I guess it’s just sad.

Normally I try to chew on an idea for a post for a few days; it lets me sort out my thoughts and form some kind of thesis. I’m totally not doing this here, though, so I should preface this with a note that I could be completely off-base. But I don’t think so.

Discussion about how we interact with computers heated up recently with the introduction of the iPad. Lots of nerdy types (myself included) were frustrated that Apple had introduced not a tablet “computer,” but a big iPod Touch. They’re both computers, of course, but the way we interact with them is different: the modern computer interface uses a multitasking windowing motif, and the iPod/iPad interface is fullscreen and single-task focused.

As a Nerdy Power User, I am well-versed in how to navigate a multitasking interface, and for the most part I understand how and why it works the way it does. I, in fact, enjoy learning about the intricacies of these kinds of systems. So when I use a single-task interface like that of the iPod Touch, I frequently bash my noggin against the barriers it imposes. Copying a URL from the web browser to my Twitter client takes orders of magnitude longer than it would on OS X or Windows, for example.

What I’ve learned from interacting with most computer users, though, is that they do not give a rat’s ass about how computers work. They want to accomplish certain tasks, and will do this in the way that is most sensible and direct for them. And the way they end up accomplishing these tasks within the multitasking window motif is typically not the way I would do it.

The recent fiasco on ReadWriteWeb, where a RWW article became the first Google result for “facebook login,” is a classic example of this. And, unfortunately, so is the reaction of most Learned Computer Fellows: one of mockery and derision, admonishing the confused users for being stupid, incompetent, or lazy.

I’ll admit that I took some glee when I first saw the numerous comments on the article; I love a humorous clusterfuck as much as the next guy. But seeing some of the reactions by the Very Smart Computer People, I began to realize that We Are Not Getting It. Consider:

• Isn’t this really a failure of Google? How did it become so easy to game search engine results that an article about Facebook and AOL became the first result for ‘facebook login,’ instead of the obvious thing people are actually looking for?

• How is it the fault of the users when we present them with multiple, barely-differentiated text fields within the same window. Is it really surprising that they don’t understand the differences between each? And is it surprising that they choose to use the one which works with more natural language, rather than entering syntactically-unnatural domain names?

• There is LOADS of anecdotal evidence that most users simply use search engines as a sort of natural language CLI. Shouldn’t we be designing interfaces that work in the way most natural for the majority of users?

These people have better things to do with their days than tweaking out the spacing in their browser toolbars. A computer for them is a utility. One that is increasingly complex, and one that is used because it’s the only option for accomplishing certain things – not because it’s a good option.

It’s kind of like the Photoshop Problem: when people want to crop a picture, we give them Photoshop. Photoshop is a behemoth application with nearly every image editing and touchup function imaginable, and it is terribly complex. Now Photoshop is an impressive tool, but only a very tiny percentage people need the power it offers. The vast majority just want to crop their ex-husband from the photo and let their friends look at it. But even iPhoto, the poster child for Apps So Easy Your Grandparents Can Use Them, continues to pile on features and complexity.

When folks need an elevator, we should give them an elevator, not an airplane. We’ve been giving them airplanes for 30 years, and then laughing at them for being too stupid to fly them right.

I think we’re the stupid ones.

Update

As I said at the start, I wrote this piece a bit off the cuff, so upon further review I think I could have made it a bit clearer. First, a couple great rebuttals I read:

• Reply to Funkatron’s Analysis of the RWW/Facebook Debacle of 2010
• No, We’re Not The Stupid Ones

I posted a comment on Phil Crissman’s blog, which I think explains a bit further what I’m thinking, and addresses the notion that some learning may still be required. To copy and paste myself:

I certainly don’t think that the computer can become (anytime soon) a magic box that determines our whims, nor do I think that people shouldn’t have to learn some things.

What I do think is that the current interface modern OSes on computers provide is simply overwhelming for most users, to the point that it’s very challenging to learn how to accomplish tasks without a very significant investment of time. Driving would be a good example of a task that does require investment of time, but is not so overwhelming that the vast majority of people fundamentally get it wrong: you don’t see people steering with their feet, or accelerating and braking with the radio. I’d argue that modern computer interfaces, in a rush to offer flexibility and capability, make it possible to steer with your hands, feet, teeth, and knees — and don’t make it particularly clear which one is best.

Update 2

Some more responses:

• Really? Are we the idiots?
• Some People Can’t Read URLs

Feel free to forward me others; I think I’ve given up trying to track them down for now.

Yesterday I put together a short survey that asked some questions about what people would be willing to pay for Spaz webOS. It’s been my plan all long to change a small fee (when possible — you can’t yet in the Palm App Catalog) for Spaz webOS, and use the revenue to support further development. Since I believe strongly that Spaz must be an open, transparent project, here are the results so far from 145 responses:

• PDF of Spaz webOS Pricing Survey Response Summary data

…and here are some comments on the results:

What is the most you would pay to purchase Spaz in the Palm App Catalog?

From doing a little math on this, it seems like $2 is the optimal price point in terms of revenue generation, followed closely by $3.

• PDF of “revenue potential” data

What 1-3 features would make you more likely to purchase Spaz in the app catalog?

First off, lots of people ignored the request to pick 1-3, which isn’t surprising (you couldn’t set a max number of choices). That being said, 4 items were far and away the most popular choices

1. Faster Performance (56%)
   Spaz does seem to perform slower than comparable apps. Part of this is because we use a combined timeline, and therefore it takes longer to build results. We also initially requested a LOT more messages from Twitter at startup than comparable apps, and processing them takes more time. We’ve lowered the default numbers (you can up them in prefs), but it’s still higher.

   We’ve also had some CPU usage issues. Those have been dramatically lowered since initial release, but there’s probably more we can do. This shows up especially while processing new messages.

   The other issue I think this response refers to is sluggish scrolling speed. This one perplexes me a bit, but I’m guessing we may be running into performance limits of the Pre’s webkit renderer. We’re going to do some experiments with stripping out CSS styling and markup on our timelines to see if the scroll speed improves in either case. If so, we will need to modify how we style and/or build the markup for our timeline.

2. Direct uploading to image hosting services like Twitpic (54%)
   This was really close to #1, which tells you how badly the lack of direct file upload APIs in webOS is hurting devs. What’s most frustrating is that the email-based workarounds actually took longer to implement than direct uploads would — we already have plenty of code from the desktop app we could have adapted. Here’s another plead to Palm to please expose the file upload APIs to 3rd-party developers ASAP!

3. New message notifications even if the app is closed (42%)
   I’ve heard this a lot. I kind of assumed people would just leave an app open if they wanted it still running, but how I use things isn’t necessarily how others will. One consequence of implementing this will be reduced battery life, so we will need to be careful about it. Note that Palm recommends not doing background network requests more than 15min per hour. We won’t exceed that if and when we do implement this.

4. Facebook support
   This one kinda surprised me. I only added it as a bit of an afterthought, but I suppose it makes sense, especially since there’s not a good native Facebook app for the Pre. I actually know very little about the Facebook API and am not super duper excited about building for their walled garden, but if someone wants to build a SpazCore lib for it, we’ll probably implement something.

Also of note were some of the “Other” responses. They included:

• “More optimized code (use less cpu!)”
  This falls under Faster perf, fo sho.

• “tweet with google maps link to current location”
  This is on the list to get implemented

• “hit user pic in timeline to @reply”
  We’re going to implement something that lets you do actions on a message without going to the detail view. Kinda sucks right now.

• “low power consumption”
  The way you do that is to make fewer network requests. This responder will not want background checking, then 8)

• “more identica/laconica support”
  I’d like to see this happen too. If you want it to happen, volunteering to code a Laconica-specific API (or somehow scraping up funding to sponsor development) would help us out a lot. We don’t have time to do much with non-Twitter APIs right now.

• “Livejournal support”
  Seems like a microblogging client isn’t a good match for this, but if we add ping.fm support, you should be able to post to LJ through that. As for reading LJ posts, I think RSS feed support would allow that.

• “It needs a non-offensive name before I would purchase it anywhere”
  We had to get one of these, although I’m surprised we only got one in nearly 150 responses. It’s not changing. Read the FAQ. Sorry!

• “Since last 2 updates it really drains my battery. LOVE it’s look and feel though.”
  These are the kinds of things we need to get reported to us. Can’t fix it if you don’t report it. Go to our support site and file an issue, please. Please make sure Spaz is the only culprit involved, though.

• “Direct blip.fm support (streaming via clicking a link)”
  That would be interesting. If someone’s willing to look into it dev-wise, I have no issue with it.

• “Why? When tweed is free. And there’s more twitter apps to come.”
  No applications are allowed to charge for apps at the moment. That will change in the future, though. I wouldn’t assume Tweed will remain free, but I have no idea what their plans are.

• “Copy and paste”
  Talk to Palm, dude. 8)

• “Change the name!!!”
  Okay, maybe we got another one, but this didn’t claim I was making fun of physically or mentally disabled people. Sorry, not changing.

• “None”
  This surprised me a little. You could have written “shits money and cleans my house.” Surely that would make you more likely to buy it, yes?

Folks who got the fancy lad Palm® Pre™ on opening day found the App Catalog chock full of superapps. One of them is Spaz, a ground-up rewrite of the award-winning desktop application I created for the Adobe AIR platform using pure HTML+JavaScript. Getting Spaz on webOS™ has been a big undertaking for me, and the process has certainly had it’s ups and downs. But I’m proud of the fact that we shipped a truly open source, transparent app on the first day of a new platform.

But much like Spaz on the desktop, this is not a revenue generator for the project. Everyone who works on the Spaz project is a volunteer, and we rely on motivated folks to help us make cool software. It doesn’t need to be a huge commitment — even just getting involved in the discussion and coming up with ideas is a big help. But here’s where we need help the most:

Coding

While I intend to be the lead on Spaz for webOS™ for a while, I really could use help — I’m working in my free time, and that’s pretty limited with a family and a day job. The full source code of Spaz for webOS™ is available on GitHub, and people who want to help make new features happen are encouraged to check it out.

Even if you’re not well-versed in webOS™ coding, if you have a good JavaScript background (or don’t and are just interested in developing one), we need help to build up SpazCore, our component library that drives Spaz for webOS™, and will drive all our projects in the future. Hacking on SpazCore requires no knowledge of webOS™.

Support

We have a brand new, very awesome support site at http://spaz.tenderapp.com/ which is sponsored by the awesome folks at ENTP. We need to direct people there, and we need to get involved in helping them there. We also should build up the FAQ/Knowledge base. In addition, identifying folks on Twitter who are trying Spaz or having issues and pointing them to the support site is very important.

Advocacy

Do you like Spaz? Why? First off, let us know — I’d like to build a repository of positive mentions like that. In addition, telling people about Spaz and encouraging them to check it out is great. Hwoever, this needs to be done in a nice, non-abusive, non-spammy way, and we always need to respect other applications and their developers. THIS IS NOT A COMPETITION — it’s making people aware of quality alternatives to closed, non-transparent software.

Design/UI

I’m an opinionated fellow and rather controlling of how the Spaz apps look, but I also know there are better designers and UX experts than me. If you’re interested in helping with design and UI work on Spaz — including additional themes — I’d love to hear your ideas.

What About the Desktop?

You are not forgotten. Spaz’s desktop client is long in the tooth, and we have Big Plans for a full rewrite based on SpazCore. That’s long-term though, and in the interim work is still being done on the current Spaz codebase, including adding multi-account support and improving filtering capabilities. We also intend to transition away from AIR to the Titanium platform.

So if you want to stop bitching in your Twitter account and actually make things better, here’s your change. Drop me a line at spaz@funkatron.com and we’ll sort out the best way for you to kick ass.

I’ve been developing with AIR for over a year now. It’s all been HTML/JavaScript apps, but the complexity of most of them has been pretty low. Spaz, the Twitter client I created, definitely isn’t simple, though – mostly because I’m kind of a crappy coder, but also because it just does a lot of stuff.

Lately I’ve been getting a little bummed about AIR, because I’m starting to bump into limitations of the platform more and more. What’s particularly frustrating is that some of these limitations are specific to HTML AIR app dev. So what follows is a couple of issues that have been particularly troublesome when developing Spaz and other HTML/JS AIR apps.

AIR gets greedy under OS X

Julien Viet, who has contributed significantly to Spaz, made a post recently about the base resources used by the AIR runtime. In it he finds that the basic “Hello World” sample app provided by Adobe used about 20MB of RAM, and hovered at about 2% CPU usage even when idling. I replicated these tests on my Macbook (2.2Ghz, 4GB) under OS X 10.5.4 and WinXP SP2, and had similar results:

I also created a Flex-based “Hello World” application to compare base usage if the Webkit renderer isn’t being used. Memory usage was slightly lower, but otherwise it was comparable to the HTML app.

The constant CPU usage seems to be specific to OS X: WinXP will jump back and forth between 0% and 2%. This is consistent with my observations of CPU usage in Spaz as well: in OS X, we get 10-15% CPU usage constantly, but under XP and Vista, CPU usage is 0% while idling. Initial memory usage is also about 30% higher with Spaz under OS X than it is under Windows. Something is goofy with AIR on OS X.

That aside, I don’t think the memory and CPU usage of AIR are surprising, considering the technology. AIR is a full Flash runtime, and HTML apps add a Webkit renderer on top of that. The memory and CPU usage is in line — or a bit lower — than most SSBs. Using traditional browser tech to build a desktop app is going to be a tradeoff in terms of memory and CPU usage, just like many other technologies that make development easier. But for one reason or another, the expectation seems to be that AIR is a lightweight technology, and it’s really not.

AIR HTML apps are leaky

What is particularly problematic is that there seem to be memory leaks within the Webkit engine as used in AIR – or at very least, it’s pretty easy to leak memory. Apps like Spaz run into this problem because it’s constantly pulling in large amounts of new data and images, and is often left running for hours or days on end.

Some of this is almost certainly because you can get away with that kind of thing on a web site, where users are unlikely to leave a single page open for days on end. Even if a web site does causes a big leakage, the browser almost always gets blamed. The libraries and tools available for HTML/JS devs, being browser-focused, typically don’t pay much attention to long-term performance issues.

Essential tools, like profilers and step debuggers, are non-existent

But memory leaks are a problem for most desktop app developers. That’s why memory profiling tools are common in desktop frameworks. And Adobe certainly understands this to be the case with Flex, as it’s FlexBuilder 3 IDE ships with memory and peformance profiling tools, as well as a full-featured debugger with breakpoints et al. That’s perfect if I was writing something in Flex, but it is completely useless for HTML/JavaScript AIR apps. I was able to get Spaz to run from within Flex Builder with a tiny bit of hackery, but the profiler gets no information about what’s going on within Webkit.

Other common choices for debugging web apps aren’t available either: Firebug is Mozilla-specifc, and the Drosera debugger and Web Inspector tools can’t hook into AIR’s Webkit instances. The information on Drosera does say:

In WebKit there is a WebScriptDebugServer which is started when WebKit starts and listens for a DCOM connection. Once a connection is made it will begin to interact with Drosera via Drosera’s ServerConnection class. WebScriptDebugServer informs Drosera about every line of JavaScript executed.

I would assume that WebScriptDebugServer exists inside AIR’s Webkit, but AFAIK this has not been utilized in any way. Adobe does have a nice AIR-specific tool called the AIR Introspector in the SDK, and it does offer some very useful functionality, but it too lacks step debugging and profiling capabilities.

That leaves us with good old console logging as our only option for debugging AIR HTML apps. That’s okay for simpler stuff, but tracking down execution bugs in more complex applications becomes extremely tedious. And console logging is basically useless for memory profiling, as there’s no insight into the underlying operations of Webkit.

I’m proud of the work I’ve done on Spaz, but as the application has grown in complexity, I’ve found the limitations of HTML app dev in AIR more and more problematic. Adobe has consistently stated that the HTML dev side of AIR is as important as the Flash/Flex side. To quote Mike Chambers:

Adobe AIR is as much about JavaScript, HTML, CSS, etc… as it is about Flash / Flex. #

If that truly is the case, Adobe needs to commit to providing a toolset for HTML/JS developers comparable to what Flex developers have. Until that’s changes, Mozilla’s XULRunner and Prism platforms are looking more and more appealing.