From technology to politics to video games; these are the random thoughts of a geek with too much time on his hands
Why the pot shouldn't call the kettle black.
Published on October 26, 2006 By Zoomba In Personal Computing

Yesterday, I reformatted my main work PC.  It had belonged to someone else before I started working here, and when I started, I didn't think to take the time to wipe it down and rebuild it to make sure there wasn't any lingering wonkiness from old applications or files.  Well, after three months of me adding and removing my own software as needed, the system reached a point where pasting an image into Photoshop to resize was a serious undertaking that pushed the machine to the very edge of its capabilities.  Now, I know Photoshop is a hog, and it's the age old joke of no matter how much RAM you have, it's only "almost" enough to run Photoshop, but this was just embarrassing performance for a Pentium D 2.8GHz with a gig of RAM.  This sluggishness showed up when doing other taxing tasks such as loading up Firefox or IE.  The hard drive was continuously thrashing, even though the disk was showing only 20% full and wasn't fragmented in the least.  So yesterday, I bit the bullet, backed up all my files to an external drive, and started over.

Now, installing Windows XP is just old-hat.  I've done it countless times on my home machines, for friends, family and in my various jobs in the past 5 years.  There's nothing tricky and I had all my drivers on a USB flash drive ready to reinstall when I was done.  I started backing up a gig of files around 1:00pm and by 2:00pm I had Windows running largely configured.  By 3:00pm I had almost all of my basic software reinstalled (Firefox, Photoshop, Office, FrontPage, IM apps, IRC etc...)  The last task was to restore my email.

Up to this point, I had been using Mozilla Thunderbird.  It's a nice, basic mail application that doesn't have any bells or whistles.  It runs well and does the email thing just fine.  However, as I find myself with more and more on my plate, more tasks to keep track of, and in general trying to keep sorted a growing amount of information, Thunderbird was showing some significant weaknesses.  It just doesn't have the additional features I needed, namely calendaring and a task list.  So, it's back to Outlook for me.  Despite my irritations with how it handles some things, and how it has the typical MS interface design stupidity, Outlook is probably the best overall information management app for Windows at the moment.

I fire up Outlook and go to import my Thunderbird mailboxes and contacts.  The contacts would only transfer by doing them as a CSV and then doing about 20min of editing in Outlook to make them work right again.  The mail itself though?  Not quite so lucky.  Turns out that Thunderbird, despite many complaints from users, has NO EXPORT FEATURE!

Yes, that's right.  A feature that every mail client I've used since Eudora 1 is missing from Thunderbird.  The ability to quickly and easily move the contents of my mail from Thunderbird to another mail application does not exist.  I have been "locked-in" to Thunderbird.  Let us pause for a moment to reflect on the irony of this situation.

Ok... moving on.

My messages are all there, I know they are, I can see them with Thunderbird.  But I can't get them into any format Outlook can import (and outlook can import a fair number of file types, and export a few too).  I know there's a way to get this all working, so I turn to Google, the repository of all human knowledge worth knowing, for the answer to this issue.  Turns out this is a very common problem people run into as I found numerous forum and blog posts bemoaning the fact that they can't get their email back out of Thunderbird in any easy manner.  A little more digging turned up the solution.  Here are the steps in a nutshell:

  1. Open your c:\Documents & Settings\%user\%app_data\Thunderbird\xxxxxx.profile\ folder
  2. Find the file names for your mail boxes, the boxes themselves have no file extension (i.e. Inbox, Sent, Trash...)
  3. Copy the folders you want to recover to another folder on your PC
  4. Rename the folders to %Folder_Name.mbox
  5. Download a freeware program called mbox2eml, install
  6. Run mbox2eml on each of the .mbox files
  7. This will result in a folder with individual .eml files for every email message in the mailbox
  8. Open Outlook Express
  9. Select all the .eml files in the folder, and drag them over the open Outlook Express window.  This will import them into Outlook express
  10. Export the contents of Outlook Express to Outlook.
  11. Cry as about a quarter of your emails are gibberish now since there is no direct translation from mbox to eml files, and some of your messages had a flaky character or bit of formatting that it couldn't handle.  If you want those messages, you'll have to read through the raw text of the message to get all the data, but you won't be able to fix the headers in your mailbox.  Those messages, for the average user, are gone.

There is absolutely no reason on earth I should have had to go through all of that to get my messages into Outlook.  It should have been a matter of exporting from Thunderbird and importing into Outlook.  Done.  No muss, no fuss.  Especially since so many people love to tout Thunderbird as being ready for the average user.  That it's just as full-featured and stable as anything you could want in that market-space.  Well, if the average user ever has to switch email clients for any reason (like lets say, their job requires it, or say Thunderbird development ever stops moving forward and the app gets abandoned), they're up a creek without a paddle unless they can find a computer geek nearby to take them through the arcane steps listed above.  Even moderately technical users familiar with the workings of import/export features in their office apps won't be able to puzzle that one out.

Ultimately though, the biggest issue I have is that so many Open Source apps tend to make it difficult to abandon them and move to an alternative product.  Settings, data files etc don't export, they're in some "open" format but no other application out there uses it or knows how to import it, or they're crippled when converting to certain applications.  When anyone points these sorts of issues out, they get one of several responses:

  • Want an export feature? Code it yourself!  It's Open Source, you have the code right there, make it yourself.
  • Why would you want to go from Thunderbird to Outlook?  Outlook is just a part of the M$ vendor-lock-in monopoly!
  • None of the developers see any reason why anyone would want to export their mail, since Thunderbird is just the best there is.
  • It's Microsoft's fault for not building a translator to import our files!

Not a single one of those is a valid or reasoned response to the request for a BASIC feature that everyone else seems to have mastered.  It's all a part of the high-and-mighty "Our way is the best way, because it's ethically and morally superior to proprietary, closed-source software" attitude that seems to permeate every nook and cranny of the open source world.  It results in its own form of "lock-in", that dreaded and evil concept where a vendor makes software such that you can not switch to a competitor without losing a significant amount of time and/or data.  Guess what?  I lost a few hours of my time getting my email sorted out because of this, and lost several dozen messages to an inaccurate conversion between formats.  Imagine the kind of cost this task would incur for any organization trying to make a similar transition... it would be horribly expensive in terms of lost productivity and data. 

But I guess lock-in is OK, if you're locked in to software that is "morally superior"

Don't get me wrong, I like Open Source Software and the ideals behind it.  I use it when it fits my needs (I love FireFox) but I'm not brainwashed on the whole absolute superiority to proprietary software thing.  If a tool provides better features, usability and performance, I'll go with that, regardless of who made it and how.  But I can't stand it when OSS pulls the same dirty tricks as closed-source software, but it's considered OK when OSS does it.


Comments
on Oct 26, 2006
Here is an idea : WHY NOT SIMPLY FORWARD THE EMAILS TO YOURSELF, THEN CHECK YOUR POP3 ACCOUNT (or imap) VIA OUTLOOK???? Your telling me that with all of your experience in computers your simply failed to realize it could be as simple as that to get your emails from one client to another? Excellent on tracking down an alternate method of doing it, and it is a very good point that thunderbird cannot export email (try "MozBackup", google it). But, you will get no sympathy for lost time/data when the answer was staring you in the face.
on Oct 26, 2006

I can't exactly forward nearly 3,000 messages (plus all the "sent" messages) to myself.  That is actually a less practical solution than what I ended up using to get my messages back.  Plus, forwarded messages (or even redirected messages) are very difficult to auto-sort or process rules on to organize.  It introduces many many many problems in organizing messages.

And yes, those 3,000 messages need to be kept.  They're only going back to mid-July too, so it's not like I'm stockpiling years of email.

on Oct 26, 2006
WHY NOT SIMPLY FORWARD THE EMAILS TO YOURSELF, THEN CHECK YOUR POP3 ACCOUNT


Have you ever tried to do this with a couple hundred emails? Have you ever noticed what this does to the subject header, that by the way needs to be edited on every forwarded email so it can comply to whatever "rules" you have inplace? Pah.

What it turns out is Thunderbird is in fact NOT full featured. Its full enough for the guy that pops an email off here and there, but hardly buisness worthy.

You get what you pay for, limited functinality and the lack of support.
on Oct 26, 2006
oops...we posted at the same time Zoomba
on Oct 26, 2006
Also, MozBackup does not export or convert mail from Thunderbird, it merely is a tool to make backups and restoration of Firefox and Thunderbird profiles easier.
on Oct 26, 2006
There is an extension for Thunderbird that will save any or all email messages as eml which can easily be dragged into any folder within Outlook Express.
on Oct 26, 2006

Yeah, the extension is a little squirrely if you try and export a folder, but selecting all the messages works fine.  I even did a search for extensions and didn't come up with anything (mostly because the description for SmartSave is incredibly vague)

Big points to Hankers for finding it 

https://addons.mozilla.org/thunderbird/2887/

A feature that definitely should be a core feature, and not a half-finished addon.

on Oct 26, 2006
A feature that definitely should be a core feature, and not a half-finished addon.


I guess that would have made T-Bird closer to "full-featured".

I learned a while back when I first started trying the various email clients out there, that Outlook is pretty much the most full-featured email client on todays market.

Now, it's all I'll ever consider using until I find something better. Why, well you just experienced why Zoomba

I paid $25 for my "new in the package" copy of outlook. I bet you would have paid $25 easily to export your email from T-Bird into Outlook quickly and easily.

One has to weigh the value of a software to himself to determine if the cost is worth it.
on Oct 30, 2006
A feature that definitely should be a core feature, and not a half-finished addon.

Then, with all due respect, code it into the core yourself, or hire someone to do it if you care that much. It is free. You get more than you pay for already. If you cannot accept the chance of it not doing what you need, then you are right, and you should use Outlook until such time as Thunderbird does everything you will ever need.

Personally, I use Outlook Express, because I found that it did all I needed better than both Thunderbird and Outlook. I may have to reconsider that come Vista, as Windows Mail no longer includes HTTP access, for no obvious reason other than that Microsoft wanted to push their Windows Live offerings.
on Oct 30, 2006
My point, which I didn't actually do a good job of articulating, was that Open Source, despite all of it's claims of end-user adequacy and inherent superiority, is not there yet. For a free mail client, Thunderbird is great, no question... but I've had to deal with too many open source people who try to tell me that it is on-par, if not superior to, any commercial or closed-source offering out there.

I've encountered similar issues and inconsistencies with Open Office and other OSS offerings. This does not mean the product are necessarily bad, just that they aren't yet competitive in terms of features to what else is on the market.

And to be honest the "code it yourself!" bit that often people do get from OSS users and developers is counter-productive if the goal is to ever supplant or at least be on equal-footing with traditional closed-source products. When it comes to software, I'm an end user. I can't code my way out of a paper bag, I know this, and accept this. However, as a user, to keep my "business", I require certain functionality. And yes, I do expect certain things from a mail client that has passed the 1.0 mark.

Open Source sits in a fun spot where it wants to play with the big boys, but still clings to the whole "you can't complain because it's free" attitude. At some point, it will have to go fully one way or another, it can't play at both ends.

on Oct 31, 2006
Yeah. Open source is not free from problems. That's why open source is not everywhere yet, but gaining popularity.

I use windows xp on my daily computer uses, and linux on my homework laptop.