I know this is a little off topic, but I'd like to tap the intellects (excuse the upcoming pun!) of the OutlinerSoftware readers concerning e-mail formats.

After Stephen's mention of Intellect I've started testing it again (I used to be an avid Time and Chaos user many years ago). I see that some of the irritants that used to bug me then are still in there (e.g. creation of all recurring tasks at one time rather than when previous occurrence is done, also meaning that all recurring tasks have to have an end date; no undated tasks allowed etc.). But one if its greatest features is the ability to run on a USB stick, which could be a great boon.

Outlook uses .pst files and Intellect .eml files to store its e-mails. A couple of questions which I can't find answers to elsewhere:

- does using thousands of separate .eml files (one for each e-mail) make a program using them faster or slower than a program (i.e. Outlook) using a single (large) .pst file?
- what are the security implications of using .eml files? I.e. I imagine that if my computer was hacked into people could open the .pst file or the .eml files to read their contents, but my gut feeling is that the .pst file is better protected. Is that a correct assumption?

Thanks in advance for any help!

Graham

Hi Graham

- does using thousands of separate .eml files (one for each
e-mail) make a program using them faster or slower than a program (i.e. Outlook) using
a single (large) .pst file?

That depends on the implementation.

- what are the security implications of using .eml files?
I.e. I imagine that if my computer was hacked into people could open the .pst file or the
.eml files to read their contents, but my gut feeling is that the .pst file is better
protected. Is that a correct assumption?

Sorry: No. Data in a PST file is not encrypted. It's not plain text either. But you can easily get software that opens PST files and lets you read or extract data.

From a security point of view, however, I'd find a collection of thousands of files (1 per email) preferable to one single file containing all your data (emails, tasks, contacts, ...). If the PST file gets corrupted, you may lose all the data in it, if one EML file gets corrupted, you lose that one email. There are, however, 3 or 4 excellent rescue tools that can be used to repair PST files or extract data.

Franz

Thanks Franz,

This leaves me in a bit of a quandary (not unusual when I have too little to do (or no mind to do what I do need to do) and CRIMP takes over).

My Outlook pst files (current and archive) are together 8 GB and, even if they weren't slowing Outlook down, they are causing disk space issues on my lap top.

I don't want to lose any historical data, but I realize I could archive older e-mails by converting them to .eml files (though I understand this detaches any attachments?) and store them elsewhere, with the added advantage that .eml files can be indexed and searched by programs such as Google Desktop, so I could still locate old texts.

Does anybody have any experience with archiving old e-mail data? Any pitfalls I should be aware of?

Graham

Graham.

Why don't you use the AutoArchive feature in Outlook to move old e-mails to an archive PST file? You can open the archive in Outlook any time you need to refer to an old e-mail.

AutoArchiving reduces the size of your PST file making it easier to handle - for Outlook and for you when backing up your e-mails.

Before AutoArchiving I would go through the e-mails - in particular the e-mails I sent - and delete the files I attached (these are usually files that are also stored in the file system because for sending them I had to save them on my hard disk). This usually saves a lot of disk space :-)

Franz

Franz,

I do that already - the archive.pst is 6 GB, current e-mails the other 2 GB. For various networking reasons the archive is also on the lap top, which causes disk space issues.

I'm just brainstorming really - wondering if there's a better way, as e-mails tend to be isolated away from other information that needs managing. Programs like Everdesk (which on my systems just explodes when it tries to import the hundreds of thousands of e-mails I have stored) tries to resolve this (by associating individual e-mails with the file system), as does Chaos Intellect (by associating e-mails with contact names) ....

In my set up it's more of a problem because e-mail is away from my main PC - so I was cogitating if at least the archive might usefully be dumped there (and be searchable ...)

Graham

The email problem can be vexatious.

My method:

1. Forward all boxes to one central gmail account. Gmail's speed, search, readability, space and hotkey navigation is unparalleled.
2. Scan emails as they come in for urgent tasks and star them if necessary for later action.
3. Batch process starred emails.
4. Extract text I will use again, then archive the email without using folders, categories or tags.
5. If I unexpectedly need to find something I didn't extract, just use Gmail's search.

Every other way I tried was an unwieldy disaster.

JBfrom wrote:

1. Forward all boxes to one central
gmail account. Gmail's speed, search, readability, space and hotkey navigation is
unparalleled.

While I do rely on Gmail, and I do find it somewhat speedy, I take some exception to the belief that its search is unparalleled. Unless things have changed, Google did not allow partial word searches in Gmail, and this is a huge problem for me. If you do not remember the exact phrase, or the exact spelling of a word or name, you are out of luck. I found this a bit unusual from the people who made a name on searches, but I guess that is how they keep their searches speedy.

--Ken

That's a good point Ken, I just realized that's been annoying me although I didn't realize exactly what the problem was.

http://mail.google.com/support/bin/answer.py?answer=7190
http://mail.google.com/support/bin/answer.py?answer=6593
It does have boolean logic.

Looks like this is the solution:
http://techcrunch.com/2011/04/15/greplins-chrome-extension-now-makes-gmail-search-infinitely-better/

You can also use Google Desktop Search.

Graham

Like you, I still need to be able to access large PST email archives.

The only difference from your setup seems to be that I store mine on an external USB drive. Access times can be demonstrably slower (especially when the drive is idle) but disk space is only limited by the size of the drive.

It's a question of trade-off I suppose. My system works for me.

Jack

Stephen's mention of Intellect I've started testing it again (I used to be an avid Time
and Chaos user many years ago). I see that some of the irritants that used to bug me then
are still in there (e.g. creation of all recurring tasks at one time rather than when
previous occurrence is done, also meaning that all recurring tasks have to have an end
date; no undated tasks allowed etc.). But one if its greatest features is the ability
to run on a USB stick, which could be a great boon.

Graham, you hit on one example of an "issue" that has lead me to use a unique feature of Intellect - the task "series." There are appointment "series," e-mail "series," etc. Check those out.

Now, that said there are irritants within Intellect for me also. All PIM's have them. There are just fewer here then in others..... or better yet the pluses of Intellect outway the minuses for me.

As for recurring tasks, I do miss Above and Beyond's handling of them. I miss the way I could track my time spent on work.

- does using thousands of separate .eml files (one for each
e-mail) make a program using them faster or slower than a program (i.e. Outlook) using
a single (large) .pst file?
- what are the security implications of using .eml files?
I.e. I imagine that if my computer was hacked into people could open the .pst file or the
.eml files to read their contents, but my gut feeling is that the .pst file is better
protected. Is that a correct assumption?

I can tell you that having thousands of eml files spread amoung numerous subdirectories does not slow Intellect down.

I can't tell you much about security. My guess and assumption is it would not matter - neither are encrypted.

Steve

Franz Grieser wrote:

Hi Graham

>- does using thousands of separate .eml files (one for each
>e-mail)
make a program using them faster or slower than a program (i.e. Outlook) using
>a
single (large) .pst file?

That depends on the implementation.

>- what are the
security implications of using .eml files?
>I.e. I imagine that if my computer was
hacked into people could open the .pst file or the
>.eml files to read their contents,
but my gut feeling is that the .pst file is better
>protected. Is that a correct
assumption?

Sorry: No. Data in a PST file is not encrypted. It's not plain text
either. But you can easily get software that opens PST files and lets you read or
extract data.

From a security point of view, however, I'd find a collection of
thousands of files (1 per email) preferable to one single file containing all your
data (emails, tasks, contacts, ...). If the PST file gets corrupted, you may lose all
the data in it, if one EML file gets corrupted, you lose that one email. There are,
however, 3 or 4 excellent rescue tools that can be used to repair PST files or extract
data.

Franz

There is a very active user forum for Intellect that has plenty of suggestions on archiving the data - email in particular. The one way I have chosen is to use the "Archive" function from within Intellect. Essentially you move the data you want to archive to a different folder - and a different user name within Intellect. Once done you can "throw" data there for archival purposes. It works for me, but there are issues to consider beforehand.

Steve

Steve wrote:

Graham, you hit on
one example of an "issue" that has lead me to use a unique feature of Intellect - the task
"series." There are appointment "series," e-mail "series," etc. Check those
out.

Ah, now that's interesting. I have a need to have a system that lists various tasks which I need to do every time something happens (such as a new country comes into being, or a postal code system changes). Currently I do that in a custom-written Visual Foxpro program, but I can see that working equally well in Intellect (except that I'd prefer the tasks to be undated, but I suppose it's just a matter of working around the inbuilt system)

I can tell you that having thousands of eml files
spread amoung numerous subdirectories does not slow Intellect down.

Thanks. I notice that Intellect reads .msg files as well, which are native Outlook files, so that might be a useful way to go as .msg files can be easily created out of .pst files just by dragging and dropping from Outlook.

Graham

By the way, the reason why I believe(d) that .pst files are more secure is that on my system (Windows 7 64-bit) Windows will stop any program which tries to read that file except Outlook and request authorisation from the user; the file is not automatically associated with Outlook so double clicking on it won't open Outlook; and its contents aren't searchable by desktop search engines.

Graham

Graham Rhind wrote:

Thanks. I notice that Intellect reads .msg files as well,

Correction: Intellect can NOT read .msg files.

Graham

Graham,

You might want to take a look at Aid4Mail at http://www.aid4mail.com/

I have used it mostly for conversions, so I haven't really tried any of the archiving functions. It can archive to zip files (and other formats) directly from pst files (even those not currently open in Outlook), so it might be worth a look.

Chris

Thanks for the GMail search links. I will check out the Chrome extension when my schedule permits.

--Ken

In addition to Gmail as a sort of backup, I would suggest using IMAP instead of POP for mail connectivity. I have taken the step with my premium Google Apps account and I must say it is a revelation. No more carrying my mail files around. I access my account via separate machines, including my Android phone and web access, and when I open my local client the read/write/flag statuses will be perfectly synced. Praise the Lord!

Chris Murtland wrote:

You might want to take a look at Aid4Mail at http://www.aid4mail.com/

Another solution is Mailbag Assistant, developed by the same person:
http://www.fookes.com/mailbag/

Best,

W.

Thanks Chris and Wojciech. I'd already looked at Aid4Mail. I'll have a play with Mailbag Assistant too.

Graham

Wojciech wrote:

Chris Murtland wrote:

>You might want to take a look at Aid4Mail at
http://www.aid4mail.com/

Another solution is Mailbag Assistant, developed by
the same person:
http://www.fookes.com/mailbag/

Best,

W.

For the sake of completeness and closure I just wanted to round this topic off with an overview of my findings re intellect versus Outlook. Obviously these are relevant to my own way of working, but they may prove of use to others when they come to look at e-mail products:

Intellect: This is a stable product but development is glacial. I was a Time & Chaos user 15 years ago and little has changed, even when there is pressure from the user base to add functionality. So you have to like this product as is and not expect any alteration in the way it works. Outlook develops mainly as a platform for business interaction rather than in core functionality, but it is already a more mature product.

Intellect revolves around contact names. Outlook can be made to do the same but in a less natural way. As Outlook will delete anything linked to other data being deleted, this tends to discourage me from linking anything just in case.

Outlook is far more flexible re tasks: Intellect demands an end date for all recurring tasks because it creates all the occurrences at the same time. It also does not allow undated tasks. Outlook does not suffer these restrictions.

Each e-mail in Intellect is stored in a separate eml file (in Outlook everything is stored in a single pst file). The eml files can be indexed for searching (whereas a pst file cannot) which gives improved ease of use against lower security.

Intellect can run from a USB stick, which is a definite advantage over Outlook. It also allows task series, which would be more useful if you could add text to them. (Currently you can add the name of a customer to them: "Send e-mail to %LINKEDNAME%") but you can't do this with another defined string, which radically reduces its usefulness for me.

Outlook has numerous plugins available and interacts with a wide variety of programs. Intellect can only interact via exported data or raw eml files.

Both programs can run various reports, but reports from Outlook can be made to look better.

Both Intellect and Outlook allow text snippets to be stored and used again in e-mails.

Intellect does not allow photographs to be added to contact records except as a file attachment. File attachments, even those for graphics, cannot be previewed in Intellect - they have to be opened by an external program.

Intellect costs about half of what Outlook costs.

All in all, for me, it's a close run thing but I'm having to stick with Outlook for the time being. If the writers of Intellect ever decide to add functionality, it would be well worth a second look.

Graham

Graham Rhind wrote:

Intellect: This is a stable product but development is glacial.
...
Intellect does not allow photographs to be added to contact records
except as a file attachment. File attachments, even those for graphics, cannot be
previewed in Intellect - they have to be opened by an external program.

These two observations are why Intellect is no longer my "goto" PIM. I've used it and T&C for years, but have returned to Outlook 2010 -- and I am really enjoying their new Social Connector add-on.

For Intellect to win me back, they would have to:
1. Allow photos to be added to contact records - I've emailed them in the past about adding this feature
2. Enable social media integration (e.g., view twitter feeds, connect to FB, extract profile images, etc.)
3. Fix and support their email client (e.g., IMAP is horrible; slow and buggy)
4. Have a development roadmap that shows they are serious about improving the product

Glen

Glen Coulthard wrote:

Graham Rhind wrote:
>Intellect: This is a stable product but development is
glacial.
>...
>Intellect does not allow photographs to be added to contact records

>except as a file attachment. File attachments, even those for graphics, cannot be

>previewed in Intellect - they have to be opened by an external program.

These two
observations are why Intellect is no longer my "goto" PIM. I've used it and T&C for
years, but have returned to Outlook 2010 -- and I am really enjoying their new Social
Connector add-on.

For Intellect to win me back, they would have to:
1. Allow photos
to be added to contact records - I've emailed them in the past about adding this
feature
2. Enable social media integration (e.g., view twitter feeds, connect to
FB, extract profile images, etc.)
3. Fix and support their email client (e.g., IMAP
is horrible; slow and buggy)
4. Have a development roadmap that shows they are
serious about improving the product

Glen

Well, to each their own Glen for what is important to that person for a PIM.

Your first two requirements are exactly the ones that would drive me away. I find those to be superfluous to a serious PIM that is as rock sold as Intellect. Those ?features? to me just waste time.

E-mail module has not been "buggy, slow, nor horrible" for me. Just the opposite, it has been fast efficient, and exceptionally stable over the last three years. When did you experience all those problems and did you report them to the developers?

Steve

Hello,

If you are looking for move your EML Files into Outlook PST File ,I can help you to do this.There are a good tool available which can very easily convert EML Files into Outlook PST Files.Kernel for EML to PST is one of them converter Tool which can convert Thunderbird, Windows Live Mail, Apple Mail, and Microsoft Outlook Express into the MS Outlook importable PST file.You can get more information from here http://www.emltooutlook.net/