Open Source Philosophy
The other day I was trying to figure out what was making my desktop computer run so slow. This was pretty unusual on Linux. Way back when I used to use Windows, I'd encounter such things due to running too many programs, so of course I looked at what was running.
But these days I don't really run that many intensive applications. I hardly ever use spreadsheets, presentation tools, word processors, or so on like I used to. Really the only things visible in my task bar were Firefox and Terminal. Killing off Firefox made no difference. And even more oddly, rebooting had no effect - within minutes my load average was up in the 20-30's.
Of course, the culprit was that I had a bunch of automatic cron jobs scheduled to occur at the same time. Reordering the timings got the system back on track.
But this led to an epiphany moment.
This was my *desktop* machine. The last time I'd seen this cron job I/O blocking madness was on servers. What was I doing with so many cron jobs on a *desktop*??
I looked at the individual cron jobs, and found that each and every one of them had something to do with my job. Disabling any one of them would reduce my work productivity. I could no longer do without one of these silly cron jobs than I could have done without Microsoft Excel a decade ago.
And then, as I looked around my desktop at the other things running on it - internet radio, IRC chat, mutt - that I noticed that I hardly ever use traditional "document-centric" applications. Everything running had something to do with dealing with _streams_ of information.
The realization sank in at that point, that our computer UI paradigm really stinks for what we actually use our computers for.
The prevailing UI paradigm today is built around the notion of document authoring. It expects that the main thing you do is create spreadsheets, word documents, presentations, and so on. There is a task bar to remind you of what documents you're editing, there is cross-application cut and paste so you can put pieces of one document into another. You can place documents on your desktop surface itself, so you can organize your work. You can define which applications to use for which types of docs. You can set up a default printer to put your documents to hard copy. You can set up system-wide fonts to use in documents. You can put icons to apps and even documents onto your panel. And on and on.
Occasionally, some of that is actually useful to me. But most of the time, for most of the work I do, it's all irrelevant. To pick one example, I used absolutely none of that in order to write this blog post.
Really, what I mostly do today is stream management. And I suspect this is true for the vast majority of people. I don't deal with writing documents, but with changes to documents. I put comments onto things. I slap patches onto things. I tweak the states of things. Once in a rare while I may author a completely new thingee, but even there I usually end up working with it as a stream of changes that I build up over time (and usually in collaboration with a few other people who stream changes to me).
Thinking about user interfaces this way, it occurs to me that there are a LOT of different kinds of streams. Streams of emails. Streams of spam to take *out* of my email stream. Streaming radio broadcasts. Streams of bug reports. Streams of software updates to apply. Streams of chatter on IRC. Streams of youtube video URL's from friends and family to check out. Streams of updates to my weather applet. Even my todo list is more like a stream of things coming in and going out, than like a static document I can print and save and be done with.
Each one of those silly cron jobs bogging down my computer were critically important for me, because THEY were the tools I used for helping me stay atop all these streams. They summarized my bug streams in various ways. They filtered my email streams into more organized sub-streams. They flagged tasks (and took care of certain tasks for me). They took care of updating the tools I used day to day.
It's weird that for all the document tools at my fingertips, it's a stodgy old *server* tool that is helps my productivity the most. I wonder how non-technical people not knowledgeable about crontab and scripting deal with such things?
This all got me to a key question... Since the purpose of our desktop UI is to make our work easier and more efficient, then if today's knowledge workers are, like me, more stream-oriented than document-oriented, then doesn't it stand to reason that we ought to re-think our UI design to optimize it for making stream management easier and more efficient? How would such optimization be done? How would such a UI look and feel? What kinds of toolkits would be needed?
There were some great comments to my previous blog entry, Pay bounties in time, not money, including this one:
I would love to pay in time and not in money. But I don't have any programming skills whatsoever. I'm an office manager by trade. I'm skilled in interactions between people and (basically) herding cats... and I'm good at it, if I can say so myself :)
How can I use this experience to help Inkscape or Ubuntu? I don't really know if I can.
For Inkscape, I've already written up an answer in depth in Inkscape's FAQ entry: Are there non-coding ways to help? There has been a huge number of non-coder Inkscapers who have gotten involved with the project in these ways and helped to make it a roaring success. So for that half of the question let me just point there, and focus on the Ubuntu side of things.
A lot of people don't realize they CAN contribute. A widespread assumption is that software is technically hard and that you need advanced coding skills to do anything to contribute. It simply isn't true. And in fact if people think that way, it holds FOSS back from its true potential. Have a look-see at the above Inkscape link to see the big breadth of non-coding things that an average piece of software needs done to make it succeed: Tutorials, clipart, teaching other users, mocking up dialog changes, translating, testing, marketing, and (especially!) helping with the bug queue. Working on any of these (especially bugs!) can have the add-on effect of freeing up those who DO have coding skills to focus them on coding instead of all the other non-technical tasks that need to be done in order to make the software successful.
Let me focus specifically on management ("cat herding") since that was the question, and try to slant it towards Ubuntu.
I suspect one of the things developers enjoy about working on open source is that they don't have a boss! I'd bet that hardly any FOSS project would say they "need a manager". However, there is more to "management" than "being a boss", and indeed I think these aspects are among the highest areas of need with most open source projects.
For these reasons, let's also distinguish between "project owner" management where you are the primary lead of the project, and "contributor" management where you're a regular team member, and narrow our focus to the latter.
With Open Source management, I like to think it involves three kinds of work beyond regular development: a) Cheerleading, b) Taking out the trash, and c) Operating the switchboard.
"Cheerleading" is essentially just encouraging good behavior and motivating and inspiring people to do more work. If you've ever worked in a demoralized environment you can know how big of a productivity difference it can make when people are happy vs. not. With volunteer projects - as most FOSS projects are - morale is HUGELY important, and unfortunately there are a LOT of demoralized open source communities out there. Getting morale back is very hard, but imagine what a difference it makes for FOSS overall to take a dispirited, nearly defunct community, turn things around, and get developers excited again about pushing their software project forward.
With Ubuntu, this could involve simply cheering folks on at e.g. bug hug days, issuing thank-you's to people who fix bugs you care about, etc. or to identifying ineffective communities around particular areas of Ubuntu you think could be better, and stimulate them to be better organized and better focused towards progress.
"Taking out the trash" is the idea of handling the chores that other people don't wish to do, yet that have begun blocking up the project. This could be helping get the indefinitely-delayed release finally out the door, cleaning up an out-of-control bug tracking system, organizing a marketing effort, researching into the background of a serious blocker bug, fixing the website, or resolving some differences of opinion that have split the project. In other words, find the things that EVERYONE complains about, but no one actually does, and to devote yourself to doing THAT. Often, once you demonstrate leadership charging right into the problem, others will follow your example and assist - sometimes even taking care of the technical hard parts you are apprehensive about doing yourself!
In Ubuntu, certainly the biggest "chore" right now is the mass of bug reports in the bug tracker. Organizing testing activities is another area where contributions could make a huge difference.
"Operating the switchboard" is probably one of the most rewarding (and unrecognized) management contributions you can make in FOSS. Everyone knows that lots of issues end up just being failures of communication. So if you turn this around, you realize that many problems can be solved just by stimulating the right channels of discussion. Thus, without really needing to deeply understand a technical issue, you can solve problems just by finding the right two people, and linking them together. For example, this might be by linking a well motivated but inexperienced coder banging their head on a bug, with an experienced coder who can give advice; or it could be putting an upstream developer in touch with an engineer at a hardware manufacturing company with the data sheets about the HW you care about; or it could be getting developers of two different applications together to discuss issues like a consistent protocol format. You know when you've made the right introduction, because you see the two instantly go off into technical depths way over your head. Be sure to check back in a day or two to see how the discussions went, so you can help see that follow up actions get taken, or results communicated to third parties appropriately.
For Ubuntu, there are probably infinite areas where locating the right upstream person and hooking them to the right MOTU or core-dev person can advance a bug a LONG way.
If you asked me what the worst day in the world is to announce an open source project's release, April 1st would be it. Yet here we are, and the announcements for 0.46 are hitting the wire now.
Actually, our true release was about a month ago, but we had to delay announcements because of a series of bad problems with the Windows package. I set an ultimatum that we would be announcing on March 24th regardless - the ultimatum worked at getting fixes in for all the issues, but we still didn't have them integrated into a working .EXE. But thanks to a bunch of new and old Windows contributors, in a week all this got put together and sorted out... just in time for April Fools Day. Heh.
Anyway, make up a Windows rant and insert it here. An apropos one for today would have something to do with open source projects not supporting monopolies that subvert ISO standards processes. But pick your favorite, I'm sure I'd agree.
Meanwhile, go get your Inkscape 0.46 from our website. There's been a year's worth of awesome features and fixes added to it since our 0.45.1 release.
As we near in on the Inkscape 0.46 release, I've been increasingly focusing on the few remaining "critical" bugs. A lot of these are specific to the Windows port, which is a bit frustrating for those of us whose big hope in life is to *replace* Windows, not to support it.
For Inkscape, popularity on Windows was sort of a side-effect not an objective. The entire raison d'etre of Inkscape (going back to the Sodipodi and Gill origins) was because of the lack of drawing tools on Linux. The proprietary Windows drawing app vendors refused to support Linux, so we all set off to fix that issue ourselves, and in so doing help make Linux a more viable alternative to Windows. For me personally, solving bug 1 is a big motivation to working on Inkscape.
In fact, while there had been some work on Sodipodi to do a Windows port, when we started Inkscape we dropped that, so we could focus on the Linux core. But it was not long before people who loved Inkscape and wanted it to work on Windows volunteered to undertake the porting work. We figured, hey, if it didn't impact the Inkscape core, and if these Windows contributors took care of all the porting work, what could it hurt to have more platform independence? These porters deserve a lot of credit - not only did they pull off a decent port, but the Windows version of Inkscape has proven to be extremely popular, and today our estimates are that we have vastly more Windows-based users than any other platform.
One could also argue that a Windows port could help provide a "stepping stone" for people to ease their migration from Windows to Linux. I have no clue whether this has come to be, nor know of any way to measure the significance of it if it has.
A point I often hear made is that having a good Windows port of Inkscape will "increase it's popularity." Perhaps, however popularity by itself isn't valuable - it needs to be translated into something tangible. For proprietary software, increased userbase means increased income, which allows hiring more developers to fix the bugs that the increased number of users are reporting. For free software projects, popularity doesn't bring value quite so directly - although you definitely get the increased number of bug reports!
In theory, you would expect that increasing userbase would result in increased number of contributers. This is the whole concept of Open Source - more eyeballs makes bugs shallow, a rising tide raises all boats, et al. I say 'contributor' rather than 'developer' deliberately, since value to an open source project can come in the form of a lot more than just code - documentation, translations, bug triaging, testing, marketing, tech support, and on and on.
So rather than just looking at userbase size by itself, consider the *ratio* of contributors to userbase. A platform with a high C/U ratio is going to have a lot more luck at stabilizing and optimizing its package than one with a low C/U. Commercial software can get by with low C/U's by paying developers to work on it; volunteer-only FOSS projects don't have that option though.
Thinking in terms of this ratio lets you think of larger problems in a bit different light. For instance, to determine if it's worth supporting a given platform, you could imagine comparing the C/U ratio to a Bugs/Userbase ratio. If C/U is too low in relation to the B/U ratio, the project won't be able to keep up with bug reports, so supporting the platform is going to literally be more trouble than worth.
This also lets you consider whether userbase growth due to increased popularity is going to be a good thing or a bad thing. If you have had a high C/U ratio previously, you can be relatively certain that increasing the userbase will bring with it a lot of new contributors. Often this can take the form of users simply helping each other out with answering questions on forums, helping with bugs, sharing tips and tricks, etc. This all represents value coming into the project, and is a strong sign of success.
On the other hand, with a low C/U ratio, increasing popularity may simply bring with it piles of bug reports and frustration by all involved. With few new volunteers coming in, existing ones risk becoming burned out, and the C/U ratio continues to decrease.
I also have a hypothesis that the base C/U ratio is platform-specific and derived from the level of technical knowledge and culture of that platform. In other words, by default, a Linux userbase will have a higher C/U than a Windows userbase, because the coder/non-coder ratio is higher, and because of cultural differences: Windows users are more accustomed to business models where they are simply consumers of products and their role ends at the point they submit a bug report; volunteer-involvement in a company's software product is not only a foreign concept but indeed something that can get you sued! On open source platforms like Linux, it's a completely different story - users are welcomed to be part of the collaborative process.
Another obvious question is, how do you improve your C/U ratio while allowing U to increase? A lot of ideas seem obvious - making the project more welcoming, encouraging people to get involved, providing ample training material, demonstrating how new participants can scratch their itches, encouraging them to learn coding, etc. Even though these ideas are so obvious, it's amazing that projects run into trouble fairly regularly because they've not done them - which can result in people not in the "inner circle" to not get involved; keeping C fixed while U goes up is just a recipe for disaster.
But I'm particularly curious how to deal with the situation of C/U on Windows. On the one hand, I don't particularly care to make life on Windows easier - I want people to switch to Ubuntu! But on the other, I want other people to take care of the Windows bugs; the existing Windows Inkscape supporters are obviously pretty overwhelmed, so really this means we need to get more Windows contributors. Telling Windows bug reporters the equivalent of "send a patch" gains either blank stares or outrage. Others would like to, but don't have the time or interest.
Sometimes, one way to increase contributions is to simply put the software bugs and all - in keeping with the "Release Early, Release Often" adage. This exposes the bugs to more people, at least some of which will be sufficiently bothered to put in the effort to fix it. But this is a mean thing to do deliberately to users.
Some would like to offer bounties, but honestly the amount of money required to compensate a developer for the time they spend fixing a bug is usually way beyond what a user can reasonably afford (and who wants to do the paperwork?) Personally I like the idea of bounty systems, but for whatever reason they don't seem to work in practice.
It's sort of ironic, that many of us got involved in Inkscape to further the aims of Open Source with the intent of getting people AWAY from Windows, yet since there aren't enough Windows contributors to cover all the problems particular to that platform, here we are with an expectation that we spend extra time supporting it. Do Not Want!
I read glyphobet's blog, itemizing frustrations he's had reporting bugs to Ubuntu. While I've not had frustrating experiences reporting bugs to Ubuntu, I've reported bugs to a lot of different open source projects, and had more than a few closed in a way I wasn't happy with, so I can definitely sympathize. I hope in Ubuntu these sorts of experiences are the exception, but certainly there's always room to improve.
In my own experience at having bugs closed in ways I thought were inappropriate or incorrect, while I was initially angry about it, after reflecting I realized in each case it was really a consequence that I didn't understand that particular project's bug philosophy and had filed the bug in a way that didn't adhere to their guidelines.
In working on X.org for Ubuntu, I've handled a LOT of bug reports, and while I personally try to be very cognizant of handling the reports in a respectful way, I've seen a lot of basic mistakes that reporters make, that will likely make it hard to get the solution they're hoping for.
So, here's my advice on how to be a great Bug Reporter for Ubuntu:
1. Attach evidence of your problem. This is the most common important thing most bug reporters fail to do. For X.org, this means Xorg.0.log, lspci output, and so on. Photographs of the screen are good too. Anything with error messages is good. The more evidence, the better.
I can easily spend a full day a week going through bug reports and requesting better evidence for bugs. If users were more proactive in providing this info, it'd allow more energy to be focused towards finding solutions.
2. Choose a good title. An ambiguous title, like "X looks wrong" is going to accumulate a lot of (false) me-too's, which will both distract from getting *your* issue solved, and give false hope to the me-too'ers who don't actually have the same issue.
Launchpad's bug reporting interface tries to match a new report to existing ones, and it seems to give a lot of weight to the title. If your title is too generic, launchpad may inadvertently direct people to your bug, when
3. Give clear steps to reproduce the issue. Some of our most frustrating X bugs get titled something like, "X crashes randomly after some period of time." These are next to impossible to investigate, and usually aren't worth reporting upstream.
4. If you have a crash or lockup, attach a backtrace. Yes, backtraces can sometimes be a lot of work to get, but honestly it's rare that action can be taken on a crash bug until we know where in the code the crash is happening; a backtrace (usually) indicates this.
5. Don't expect Ubuntu will fix the bug. Ubuntu has a *huge* userbase and while we have a lot of good developers, the rate of bug reporting far, far exceeds our bug fixing abilities. Even a perfectly reported bug can often take several days or a week to create a solution for; for a popular Ubuntu package (xserver let's say) it wouldn't be unusual to have a dozen or two new reports come in over that same time period.
So, in many cases, the best outcome you can hope for is that your bug gets reported upstream. You can often have faster results by shortcutting this process and forwarding your report upstream yourself. Be aware though that upstream projects may have stricter (or different) bug reporting guidelines.
Sometimes bugs really are specific to Ubuntu, and it's these bugs that Ubuntu engineers try to prioritize their time for; obviously if the issue is our fault, we need to put priority into fixing it. We'll also give priority to bugs that are severe and/or widespread, or that look relatively easy to fix. The former is usually out of your hands, but the latter you can do something about:
6. Make your bug easier to fix. Even if you've followed all the above steps, you still may not see progress on your bug, but that's not the end of the story! If you have time and patience (or are sufficiently driven by the frustrations), there's several things you can do to help improve your bug's chances:
a) Vary some parameters. Try reproducing it on different hardware, or different OS's, or different Ubuntu releases. Try reproducing the issue several different ways.
b) Find dupes. Often other people will report the same issue, but maybe in different ways. Also look upstream and at Debian's bug tracker. Dupe bugs are valuable because the other reporters often have interesting insights. As they say, "Many eyes make bugs shallow."
c) Seek out patches. If you find a fix somewhere - forums, upstream bug trackers, casual acquaintances, whatever - these can often turn the bug into very low hanging fruit. The developer can then just doublecheck that the patch won't cause issues for other people. Even negative findings can provide valuable clues.
d) Be active with your bug. Many bugs cannot be reproduced by the developer, so they rely heavily on the reporter to carry out experiments, test patches, and so on. If you don't think you have the technical skill to do this, say so and ask for pointers or directions.
As I mulled over some of our recent packaging issues with UME's X components, it struck me that in Open Source there's (at least) two drastically different approaches for handling dependencies.
It's really common when developing some bit of code to learn that a fix for some issue you've run into is already available in a newer version of the library, kernel module, xserver, or whatever that you're developing against. What do you do?
The first philosophy is to immediately upgrade to that version of the dependency, and shift your program's requirements to require it. From your point of view, this makes the most sense - you solve the problem you're having with a minimal amount of time investment on your part.
However, the trade off is that it shifts the time investment to your code's users. If they wish to use your code, they find they must upgrade to whatever versions of dependencies you're using.
Back in college when I first learned about Free Software (the term "Open Source" didn't exist way back then!), one of the things that drew me to it the most was the idea of Open Source games.
In fact I spent quite a bit of time myself tinkering open source game development of various sorts in those early years. I still do from time to time. But mostly I just think about how much better the current situation ought to be.
As many, many others have pointed out, games are a key need for Linux to succeed as a desktop operating system. I also suspect Open Source games in particular are going to be vital for ensuring the long term success of embedded Linux, since so many such systems include at least a few games and since the manufacturers aren't going to want to pay commercial licenses for them.
I think this is my favorite article of the week: Giving people Ubuntu envy.
So the thing Beryl is most useful for is in winning over Linux converts. Well, okay!
I think coupling easy Beryl/Compiz installation with an extremely reliable projector auto-detection would be one of the absolute best ways to sell Linux. If you could pop a Ubuntu CD into a laptop, hook it to any random projector, and boot into Compiz/Beryl in a couple minutes, the whole conference market would be sewn up in months.
Glyn Moody gives some insights into
MS patent FUD. Compared with proprietary software, Open Source by its nature is free of patent issues, but Microsoft is attempting to muddy the waters, insinuating that there are patent troubles. The truth is, if there ever *are* patent issues, Open Source will rip out and replace the problematic code with new, better, freer code before you can blink. This is what has always happened whenever code turned up with patent problems. This is what brought us PNG, OGG, ODF, and more.
Six years ago, I was living down in California and I remember Dad sending me a clipping from the Oregonian about this new "Open Source Development Labs" starting up in Beaverton. Working on Open Source sounded like a dream job to me.
I remember my first day vividly. I rang the front bell and waited. There were only 6 employees (I was lucky employee #7): a handful of sysadmins and a manager. Christine came, opened the door to let me in, and the manager handed me an access card. Policy was fairly minimal. "Pick a cube, set up your computer yourself. Install whatever you want, as long as it's Linux." The floor had space for about 40 people, but it was almost entirely empty. There was a computer lab attached; the main occupation of all the employees was putting computers into the lab.
