Lately I've been struggling with the fact that it may be impossible to really organize personal information. ;-)

The sheer volume of things I have to deal with and the questionable benefit of putting it into some kind of well-structured environment makes me think the best approach may be to just not try to structure it. Using one of the desktop search programs, you can pretty much find what you need as long as you export any proprietary formats to something the search apps can index.

I think Ultra Recall is sort of like my ideal file system - if saved search and metadata features were just built in to the file system (and they eventually will be, I think), I don't know that I'd ever need to build a tree to attempt to organize stuff. I do have a pretty extensive tree hierarchy built in UR, but 90% of the time, I am just searching through UR to find a particular item or set of items. I rarely just browse the tree to see what's there. So what good is the time I spend setting up and organizing the tree? The biggest benefit with UR is being able to tag diverse types of items (email, documents, web pages) with a single system; that helps a lot, but perhaps I should just dump everything in one folder in UR and not worry about organizing (beyond saved searches based on the tags/attributes I have set)? The only problem with just using files and not software like UR is that there is no consistent way (that I know of) to tag or set user-defined attributes on files (with text files, I can just put the metadata in the file, but there are other file types where this wouldn't work).

Anyone else run into this problem or have thoughts on why organizing is or isn't worth the time?

Chris

thoughts on why organizing is or isn’t worth the time?

It's true that using Spotlight searches or Opal's search field finds most stuff for me. But just last night I was searching for something, and I couldn't remember quite where I put it. Or exactly what I wrote. I thought it had to do with "land" and/or "defense" which are rather common words. I simply wasn't finding my notes (even though I knew which file system folder they were in).

Finally I ended up skimming some of the outlines, expanding the major topics that might be relevant, and was able to see the orginal note as a subtopic.

So while too much attention to organizing or categorizing everything is probably wasted, I think it would be false economy to do none. If I didn't have a file system folder hierarchy the search would have been impossible. And logical topics and subtopics were also a help.

Of course, organization can also help you create notes in the first place, by helping to organize your thoughts. That's a major part of what outliners are all about...

Thanks, I think I just needed some validation that I haven't been wasting ALL my time for the past ten years.

One thing I did overlook while thinking about this is that even after locating an item, having it in the outline structure also lets you see it in context with other related items that may not exactly match the search.

I guess the thing I need to work on is defining the conceptual boundaries of where to stop building structure because it won't be used and where it is really useful. A lot of reference type information seems to lend itself to just needing to be able to be located by a search, but active ideas/thoughts and writing are where outlines really help me. And in this case, the tree-based database approach seems less helpful than the one-pane outline approach (where the text *is* the outline item instead of a separate note attached to the outline item). So it looks like I need a more traditional outliner for active thinking and a database approach to storing reference information. Ecco or BrainStorm on Windows would probably be my top choices for the former. (I'll be sure to try Opal if I ever get a Mac).

It's sometimes hard for me to make distinctions between ideas/actions/reference material; they seem to shift and morph into each other along the way. I suppose that's why I still lean toward trying to do all my info management in one app. For all its crippling flaws, one thing I did like about Info Select was that you could kind of effortlessly switch between tree-based database type info and fairly decent one-pane outlining all within one tree.

Chris

Chris, I find it impossible to structure/organixze my thoughts, interests, projects, blah_blah. If I tried it, I'd be spending time doing that rather than following my intersts or accomplishing anything so I've always searched for programs that could absorb random info & then organize it for me.

To that end, I started with Lotus Agenda, then switched to EccoPro (when, sadly, Agenda was abandoned)
Then I revisited Zoot (direct descendant of Agenda) but it's in the process of being coded for 32bit

For the past few years I've been using Ariadne & ADM. (basically). I just dump everything into ADM & use it to gather data when I need it. It makes me feel warm all over when I see the results. Both these programs are a little quirky but affordable & in continuing development... albeit not always as quickly as one might like.

Daly & a few others can talk about both these programs as well. I'm not touting either. I'm just telling you what I use for the exact same reasons you're concerned with.

--
Jan Rifkinson
Ridgefield, CT USA
skype janrif

I'm toying with the idea of going to a chronological filing method with tags (meta-data). That is, filing by date and adding tags (meta-data) to aid sorting/searching. Has anyone else tried this?

For years I filed by topic. Then, like most, found this produces..

(a) too many folders to know where any particular item is, so you wind up doing searches across multiple folders to find what you need anyway; or

(b) the never ending cross-filing problem...that any given item could be filed in any of a half dozen different topics, meaning you need to either store multiple copies, or try to put shortcuts or something to that effect in the alternate locations pointing to the one real item (waaay too much work); or...here it is again....just wind up searching for what you need anyway! (there's a theme here!).

So then I took to storing everything in one folder with tags (meta-data, categories, whatever-you-want-to-call-this-sort-of-thing...)...and filtering/sorting/searching to find what I want. This works better, but still can take too long to find something because much of my work has all the same words in it (and the same tags or meta-data)...so searching returns too many items with no ability to discriminate between them. For example, my team at work and I are constantly exchanging e-mail with data attached...but all the messages have the same recipients, many of the same words, they're about the same topics, and they have similar data attached. How do I find stuff in that pile???

Sorting through the results of all these searches showed a new theme...and that was that I often could remember the period of time when the item was filed. Hence my idea that I would give up on the everything-in-one-folder-with-tags approach and go back to filing, but by chronological folders.

Has anyone tried that?

TIA,
--
dgg

Jan,

Thanks for the ideas. I was pretty excited about ADM at first, mainly because it functions as a both a real outliner (paragraph text within the topic) as well as a database at the same time. However, I'm one of those who has soured on it a bit because of bugginess, data loss problems, etc. I haven't tried the latest couple of releases, though. I've also tried Ariadne in the past, and like almost everything else, it has some appeal to me.

I think the app that I like the best for just shoveling random info in and then making sense of it later is Zoot. But I guess I'm ready for it to modernize a bit, even though it is very powerful as is.

But I think I'm beginning to see that the mechanics and details of anything are probably not going to totally satisfy me any time soon, so I'm looking for conceptual ways to segment different types of information into the apps that suit them (random piles over here, outlines over here, grids over here, maps/visualization over here). The trick here is that it's actually pretty hard to make things separate.

Chris

dg wrote:

I'm toying with the idea of going to a chronological filing method with tags
(meta-data). That is, filing by date and adding tags (meta-data) to aid
sorting/searching. Has anyone else tried this?

Sorting through the
results of all these searches showed a new theme...and that was that I often could
remember the period of time when the item was filed. Hence my idea that I would give up on
the everything-in-one-folder-with-tags approach and go back to filing, but by
chronological folders.

Interesting idea, and I think the intersection of chronology and category/type/tags would provide a pretty robust retrieval mechanism. Question is, how do you plan on "tagging" your files? Just in the file name?

Chris

Jan, I'm going to post something on Ariadne later tonight or tomorrow morning.

I am putting together a grant proposal for a pilot program to treat people with borderline personality d/o using a new therapy approach. What I have done is develop a writing template using Ariadne, and taking advantage of its comments, notes, particles, and colour highlighting of outline items.

It's low tech -- but appropriate tech, if you know what I mean.

More later.

Daly

Jan Rifkinson wrote:

Chris, I find it impossible to structure/organixze my thoughts, interests,
projects, blah_blah. If I tried it, I'd be spending time doing that rather than
following my intersts or accomplishing anything so I've always searched for
programs that could absorb random info & then organize it for me.

To that end, I
started with Lotus Agenda, then switched to EccoPro (when, sadly, Agenda was
abandoned)
Then I revisited Zoot (direct descendant of Agenda) but it's in the
process of being coded for 32bit

For the past few years I've been using Ariadne & ADM.
(basically). I just dump everything into ADM & use it to gather data when I need it. It
makes me feel warm all over when I see the results. Both these programs are a little
quirky but affordable & in continuing development... albeit not always as quickly as
one might like.

Daly & a few others can talk about both these programs as well. I'm not
touting either. I'm just telling you what I use for the exact same reasons you're
concerned with.

--
Jan Rifkinson
Ridgefield, CT USA
skype janrif

Chris:

Re "The only problem with just using files and not software like UR is that there is no consistent way (that I know of) to tag or set user-defined attributes on files (with text files, I can just put the metadata in the file, but there are other file types where this wouldn’t work)."

Have you seen "My SmartFolders" - http://www.castlepeaksoftware.com/Products.html ?

Regards,

Ian Goldsmid

Ian, thanks for the tip. That looks interesting indeed. I will try it out.

Chris

how do you plan on "tagging" your files? Just in the file name?

Chris

I use the "Comment" field on files to attach meta-data.

I use xplorer2 as my file manager http://www.zabkat.com/ which has a hot key (Alt-z) that allows a quick pop-up dialog to edit the Comment field. This can be done from Windows Explorer, too, in the File Properties box, but it requires a few more keystrokes.

I then use MS Windows Desktop Search (WDS) v02.06.5000-5401 to search on the Comment field -- i.e., "comment:blah".

Very quick and fast.
--
dgg

Chris, At the end of the day, you are probably right stating that there is no one program to do it all.

But having said that trying to parse the "type" of information & to find an appropriate program to handle it seems self-defeating to me because, IMO, that's what currently exists by an large.

However, I often tried to explain my different uses of Ariadne & ADM in this way. For day to day use, i.e. todo lists, AB, calendaring, etc. I use Ariadne while I use ADM for data gathering, primarily from external sources -- email & websites.

So for me, I think the natural line of diviision when it comes to information has been static/self generating info goes into Ariadne while externally driven info, i..e email & webhosting goes into ADM.

I admit both these programs are a bit buggy at times though I think both are making progress in that dept and I must say, despite the bugs here & there, I have never lost any data over the years via either of those programs.

Like you, I think Zoot is ultimately ia the most powerful data manipulator out there but, even in its sophistiction, it's in the stone age IMO & I don't know how long/if it will catch up with today's expectations. OTOH, I will keep my eye on it.

I think I've tried most of them & I'm willing to jump to something better but for the time being, I'm pretty satisfied where I've come down in this dept although I keep looking..

I think a lot of this discussion depends on where you come down on structure vs unstructured data at the entry point.

Here's an example. For my digital photography I use a program called iMatch to keep track of my +/- 17,000 images. The beauty of this program, IMO, is that you can be a specific in cataloging your images as you want to. Some of it is automated but you can categorize & gather to your hearts content. It depends on what suits you. This is what I call un-structured data entry. Same goes for ADM.

On the flip side of that is a program like Outlook which requires the user to adhere to a strict format while it's being entered. The more the user has to think abou the format, the less useful I think the program is.

So you might find some answers by trying a different perspective.

--
Jan Rifkinson
Ridgefield, CT USA
skype janrif

Jan Rifkinson wrote:

Here's an
example. For my digital photography I use a program called iMatch to keep track of my
+/- 17,000 images. The beauty of this program, IMO, is that you can be a specific in
cataloging your images as you want to. Some of it is automated but you can categorize &
gather to your hearts content. It depends on what suits you. This is what I call
un-structured data entry. Same goes for ADM.

I am also an iMatch user (http://www.photools.com well I've dabbled in it, but have not gotten around to devoting full attention - keep getting sidetracked by something else.

This is something new to the latest version - course iMatch is not a writing tool.

From their Supported File Formats section:

Document, Office and Multimedia Formats

Although IMatch is focused on digital image management, it provides support for a wide range of non-image formats too. For many formats IMatch can even render thumbnails or previews7. For other file formats, IMatch uses the standard icon representation you know from Windows Explorer.

All file-management features in IMatch (including Copy/Move/Rename and Categorize) are also available for non-image formats.

Adobe PDF (PDF)
Microsoft Office (DOC,XML,PPT,MSP,...)
Multimedia-Formats (AVI, MOV, RM,MPEG,MPG,...)
MP3 Audio (MP3)
Encapsulated Postscript (EPS)
Others can be configured as needed


Later,
Ken Ashworth

dg, how does this file manager compare with other programs out there?

Does anyone know whether it is better than IDEA, or whether its functionality is something much different?

Daly

dg wrote:

> how do you plan on "tagging" your files? Just in the file name?
>
>Chris
>

I use
the "Comment" field on files to attach meta-data.

I use xplorer2 as my file manager
http://www.zabkat.com/ which has a hot key (Alt-z) that allows a quick pop-up dialog
to edit the Comment field. This can be done from Windows Explorer, too, in the File
Properties box, but it requires a few more keystrokes.

I then use MS Windows Desktop
Search (WDS) v02.06.5000-5401 to search on the Comment field -- i.e.,
"comment:blah".

Very quick and fast.
--
dgg

Daly

Idea! isn't really a file manager. It is more in the same class as UltraRecall i.e. a data management app (at least for me).

I use PowerDesk as a my Explorer and have done so for many years. It does hav an ability to add file notes, but it is functionality I don't use.

I think we have discussed this before, so if you search those wonderful archives for PowerDesk or ExplorerPlus you should get something.

Jack

I have written a blueprint for a software that addresses this issue... a tag-based PIM.
It's on my blog: http://indiegeek.blogspot.com/2006/08/taglogger-idea-for-tag-based-pimgeneral.html
I call it Taglogger (or) Notarius. Unfortunately i'm not a programmer myself, so although it's pretty detailed (you can tell it by the size of it), it's still just an essay about my dream-PIM. I hope that i can grab some developer's attention and this may come true someday. Recently, i have received an email from a programmer on Norway which may try to implement it for the mac, but he doesn't have much free time so i don't know if it will get real.

If you take the time to read it, i'd very much like any comments on it. Thanks,
-Paulo

Chris,

You are raising the right issue:
I always think that I should remove all the Outliner/PIM/Organizer software from my PC and just use a simple text editor and save all information, snippets, URLs etc. in one folder (or only a few folder, but not in a hierarchy (folder in folder)). on my USB stick. Then I just need a very simple and fast text search engine to retrieve the needed information in these text files. Most text editors have such a search engine included. Best is, when you can save your search queries for late reuse.

I think it is worth to test such a basic approach. If you don't want to use a text editor your can implement this approach in Ultra Recall, ADM or Zoot (or any other application).

These are my main issues with hierarchical organisation of data:
- I don't know anymore in in which folder I have put my note three days ago
- The created hierarchy today is logical and helpful. But in two weeks I think 'why have I created this stupid hierarchy?'
- To find an information where I don't know exactly where it is I am using the search engine

Dominik

While i agree that scraping the outline is the way to go (for all the reasons you described, and also because they tend to get huge with time, so even if i remember where i put such info, it still take ages to navigate through the tree and all its levels), and also that being able to search all the data and being able to filter and actually find stuff in a snap is a central feature of a great PIM, I think that while searching and finding works when you need to mine a specific information, sometimes you want to see things information in a context, alongside other bits of similar information, so there is a big room for some kind of organization.

So, instead of the hierarchical approach, which is deeply flawed in a information overload scenario (good for digesting information, but not for capturing and making sense of it in a quick way), i think that the core organization feature of a PIM should be tagging (metadata). Tags are good because you can classify stuff in a very direct, cognitive way, which is designating categories to the information on an equivalent way your own brain would label the same information. It is very convenient too that you don't need to put that new info in the structure of your universe of information. You just capture it, give it some tags and done. Tags also don't have parents or children. They don't have levels. They just intersect with each other, which is a great way of filtering similar information, and viewing it in ways that you'd never imagined when you tagged the info. It subtracts from the cognitive cost of having to choose a single place on the structure to place the info. With tags, an info can be on many places at the same time.

My ideal software for handling information would be such a PIM, with an 'capture agent' sitting unobtrusively on background, just waiting to be called by a keyboard shortcut (or by clicking on an icon at the tray) for new information to be filed. If you have a selection when you call it, it is acknowledged, and you just need to tag it and make minor edits to the text. If you don't have a selection, you can always call the capture agent, type the new entry manually and then tag it. It can be anything, a phone number, an URL (del.icio.us style here), an inspiring prose, etc. It's all stored in the same place, and at the same time it is on a multitude of different places, because you can always choose to just see a particular tag on the main window. This can be very good to implementing GTD contexts.

Speaking of the main window (and since the thread starter asked for our thoughts on the subject) i think that my ideal PIM would be very similar to a web browser. But instead of an address bar, you have a tag bar, on which you can enter the desired tag or tag intersection to enforce. Similarly to an Internet URL, each and every possible tag or combination of tags could be treated, as a matter of fact, as different locations, each one being customized for displaying its data on different ways. So, in some, you can just have your entries as a simple list on reverse-chronological order, like a blog. On others, you can display your entries alphabetically. Yet, for another tag or intersection of tags, you can create 'specific data fields' for the entries that bear those, and having the respective location to show the entries on a grid, just like a spreadsheet, with a variety of ways of sorting the grid based on the specific data fields of the entries on that location. This way, one could create specific mini-databases inside the general purpose knowledge base. And better, if you are filing a new entry in the 'capture agent' described on the preceding paragraph, and you manage to tag an entry with tags where you have created specific data-fields, the software could acknowledge that and let you fill all of those fields on-the-fly, just because you have tagged the entry on a particular way. (Zoot meets del.icio.us?)

Not letting the webbrowser analogy go, it would also be good if the user could bookmark some locations (tags or combination of tags), so he doesn't need to type an often-used long tag intersection every time he needs to reach it. He could even make an outline hierarchy out of his bookmarks if he wanted, and this, IMO, would be acceptable because each bookmark would really act as a smart folder, and not like a regular node on a hierarchic structure. The cost of deciding where things go would still be low because it would still be all about what tags you do give to each entry, and not to where you put the new info on a huge structure. Of course, outliners are great for their own stuff, mostly when you want to structure a specific subject and can expect the outline to reach a given size and not grow beyond that. But i don't think you can manage all your life's information on an outliner.

Paulo, there is a program that does a lot of what you describe on your blog.

It is called MDE InfoHandler. InfoHandler is all about tags and tag combinations.

Anyone who is into tagging to any extent needs to look at InfoHandler.

IH frees you from dependency on a hierarchical tree structure -- yet if you want an overview of your tags it can be presented in a convenient tree-like form.

What I like about IH is the combination of categories or tags.

In some programs if you had an article about Freud you might assign the tags Freud, unconscious, seduction.

But later you could not do a search that would bring up only those articles about Freud that dealt with the unconscious and seduction.

IH allows for unique search combinations that can be very, very specific based on the tags used for a search.

As well, IH has the normal search capabilities, including the ability to do line or page searches where the actual context of the found search words is shown.

IH is a very neat program. Given all the talk these days about tags, I find it interesting to realize just how far ahead of its time IH is, as well as the fact that people often dream about an ideal program, not realizing that something similar already exists.

IH is at http://www.mdesoft.com/eng.htm .

Daly

Thanks Daly, but i already knew MDE Infohandler.

It is a huge of a job considering that the author does it alone, but i really don't like the interface...
I have a reason for that. It's crowded and cumbersome. Changing from hierarchical to tagging isn't only because you can intersect tags, but mostly because tags have a low 'cognitive' cost, so to speak. If the interface makes it hard instead, you lose a big reason for using tags.

Maybe i just didn't get it the first time. I'll download it and test it again with more patience... Thanks

Daly de Gagne wrote:

In some
programs if you had an article about Freud you might assign the tags Freud,
unconscious, seduction.

But later you could not do a search that would bring up only
those articles about Freud that dealt with the unconscious and seduction.

Daly,

A simple program like Personal Knowbase can do this very thing quite easily. You just assign those three keywords to your article (and any others you would like to add to it). Later, when it is time to search, just select those three words from your keyword list and you achieve the same result.

The problem with PK is that your list of keywords quickly becomes overwhelming... usually longer than your list of articles. The only way to combat this is to use different databases for different purposes, but that, of course, means you have to remember which database you want to search in for a specific item, as PK does not provide for a cross-database search. I'm not an IH user, but its advantage seems to be giving you ways to better organize your lists of keywords (or in IH parlance, categories) so they do not overwhelm you. Still, as Paulo indicated, I become overwhelmed just looking at the IH interface.

Steve Z.

Paulo, I agree that the interface takes some getting used to.

However, Manfred has recently made what I consider is an elegant improvement by the addition of a category tree on the left side of the window -- thus evirtually eliminating the need for the bottom section of the window where traditionally one has assigned categories.

I have found that the more I use IH the more ease I feel with the interface.

I would respectively disagree with your comment re tags -- for me the big advantage of tags is being able to intersect them, as well as the fact that you no longer have to think in terms of exactly where do you place an item in a tree.

Daly

Paulo Diniz wrote:

Thanks Daly, but i already knew MDE Infohandler.

It is a huge of a job considering
that the author does it alone, but i really don't like the interface...
I have a reason
for that. It's crowded and cumbersome. Changing from hierarchical to tagging isn't
only because you can intersect tags, but mostly because tags have a low 'cognitive'
cost, so to speak. If the interface makes it hard instead, you lose a big reason for
using tags.

Maybe i just didn't get it the first time. I'll download it and test it
again with more patience... Thanks

PK is an interesting program. But you are right about the keyword list getting too long.

IH deals with this by allowing you to group keywords or categories into groups.

As well, IH has some interesting features such as parent/child and master/slave categories that really facilitate the assignation of cateogries.

IH also lets you search across data bases.

It's of interest to me that the two programs whose interfaces intimidated me for such a long time -- IH and Zoot -- are the two programs I most use now.

Daly

Stephen Zeoli wrote:

Daly de Gagne wrote:
>In some
>programs if you had an article about Freud you might
assign the tags Freud,
>unconscious, seduction.
>
>But later you could not do a
search that would bring up only
>those articles about Freud that dealt with the
unconscious and seduction.


Daly,

A simple program like Personal Knowbase can
do this very thing quite easily. You just assign those three keywords to your article
(and any others you would like to add to it). Later, when it is time to search, just
select those three words from your keyword list and you achieve the same result.

The
problem with PK is that your list of keywords quickly becomes overwhelming...
usually longer than your list of articles. The only way to combat this is to use
different databases for different purposes, but that, of course, means you have to
remember which database you want to search in for a specific item, as PK does not
provide for a cross-database search. I'm not an IH user, but its advantage seems to be
giving you ways to better organize your lists of keywords (or in IH parlance,
categories) so they do not overwhelm you. Still, as Paulo indicated, I become
overwhelmed just looking at the IH interface.

Steve Z.

Hi Stephen,

I don't think that that is the only way to deal with a big number of categories. In del.icio.us, i can quickly tag stuff even though i have more than two hundred tags, because when i'm typing the tags i want to use, there is a guessing mechanism that points to alternatives of possible tags considering what i have already typed and my pre-existing tags. Just as Google Suggest (http://labs.google.com/suggest does. Also, when i'm trying to find back stuff, i can type directly the tag (or intersection of tags) i want to reach, and this makes me avoid the long list of tags (even though there isn't a guessing mechanism for this on del.icio.us, it's still much easier). On IH you can't do that.

When you create a category group (i.e food) on infohandler, and put some categories inside of it (let's say: meat, vegetables, cereals, fruits), just to breakdown an otherwise very big category list, you're losing some value for intersecting categories, because without category groups, 'food' itself would be a tag/category, and not a category group. And you can't intersect category groups, just tags/categories.

I also don't like the idea of having different files with different categories for different purposes, because the PIM that someday i hope to use is for storing a general knowledgebase, and this would be incompatible by having multiple files. And yet, you lose the ability to intersect in this case.

Stephen Zeoli wrote:

The
problem with PK is that your list of keywords quickly becomes overwhelming...
usually longer than your list of articles. The only way to combat this is to use
different databases for different purposes, but that, of course, means you have to
remember which database you want to search in for a specific item, as PK does not
provide for a cross-database search. I'm not an IH user, but its advantage seems to be
giving you ways to better organize your lists of keywords (or in IH parlance,
categories) so they do not overwhelm you. Still, as Paulo indicated, I become
overwhelmed just looking at the IH interface.

Steve Z.

Paulo, you write:

"When you create a category group (i.e food) on infohandler, and put some categories inside of it (let’s say: meat, vegetables, cereals, fruits), just to breakdown an otherwise very big category list, you’re losing some value for intersecting categories, because without category groups, ‘food’ itself would be a tag/category, and not a category group. And you can’t intersect category groups, just tags/categories."

I think IH gives you a way around that problem. If you want food to be a category, you can have it as a category. And if you want anything you categorize that is a food to show up under food as well as categories you may have for vegetables, meats, fruits, home grown, easy-to-cook, etc. you just make the food category a slave to each of those categories. The when you categories chicken a meats it will automatically show up as a food also.

The slave/master and parent/child category system Manfred provides for IH allows some really neat tricks with categorization.

Daly