For anything I might make notes about, I probably have pertinent files stored in folders on my hard drives. So if I'm looking for something I might have to do at least two searches to find something I'm looking for - a search of my notes database and a search of my file system. I've long wanted to have my my file system somehow integrated into my notes database, so that the contents of my files will also show up in a search of my notes database. Of course I can manually put links to files in my notes database, but I would like for my notes program to monitor specified folders, find new content, and integrate it into the notes database.

I would be very interested to hear how other people pull relevant file contents into their notes databases, and what tools might do some part of this work.

RightNote lets you have Attachment note types (the document is copied into the database) or Link note types (the document is just referenced by the link). In both cases, the contents of supported documents (txt, rtf, htm, html, doc, docx, xls, pdf) are indexed and searchable. I haven't used these myself, so I'm not sure how it handles an indexed Link type when the external document changes.

Answer from Rael (RightNote author): If you open the file via RN it will monitor changes for a link or attachment, however it won't monitor changes made outside of RN.

jimspoon wrote:

So if I'm looking for something I
might have to do at least two searches to find something I'm looking for
- a search of my notes database and a search of my file system.
I've
long wanted to have my my file system somehow integrated into my notes
database, so that the contents of my files will also show up in a search
of my notes database. Of course I can manually put links to files in my
notes database, but I would like for my notes program to monitor
specified folders, find new content, and integrate it into the notes
database.

There seem to be several problems implied here. Which one is your priority?

1) Is the main problem that your linked files are not searchable (indexed) from within your notes database?

2) Or is the problem that you'd like to automate the linking of your files in your notes database (maybe on the basis of some rules, or all of them all the time)?

3) Or is the problem that the manual linking of files to notes is too cumbersome in your current notes database?

In terms of setup, one decision might be whether you want your notes database to be a holder of curated links (which would mean that you'd still need a separate desktop search solution for searching the contents of non-linked files, if and when needed), or a hold-all database, a de facto mirror image of your document folders.

Very good questions. One can imagine various degrees of integration of "notes" and "file content". For me, the more integrated, the better. So that when issues a query, the results may contain content from a notes database, and content from files, mixed together and organized in any number of different ways. The information manager or outliner would be able to access the content of external files very quickly - perhaps by taking advantage of an index maintained by another program, perhaps Windows Desktop Search.

(Just as an aside, Deskrule by the author of xplorer2 does not itself index files, but uses the Windows Desktop Search indexes.)

Maybe the ultimate in what I am thinking of is a merger of an outliner / info manager with a desktop search program. So that, not only can one retrieve content of external files, but one can then structure, categorize, that content.

But even if an outliner falls far short of this kind of integration I would still be interested in hearing in how different outliners can access and perhaps even manipulate the content of external files.

@jimspoon wrote

So that when issues a query, the results may contain content from a notes database, and content from files, mixed together and organized in any number of different ways.

I have always liked this concept, though devilishly difficult. Results that search "content" of files depends on file type of course. Over here that would be, mainly, plain text, Word, and PDF. On the OS X side, DEVONthink's so-called "AI" is maybe 60% of the way to @jimspoon's vision. (It's not really "AI" -- just a proprietary dataset indexing a concordance drawn from document text content.) Though, DEVONthink seems to be on a slow downward spiral -- very little advance in features for years and a partially updated UI that when released (if ever) will be nearly a decade in the making.

I think, if hints on the preview forum are accurate, that TheBrain v9 will provide better search of "notes" and attached files.

jimspoon wrote:

Maybe the ultimate in what I am thinking of is a merger of an outliner /
info manager with a desktop search program. So that, not only can one
retrieve content of external files, but one can then structure,
categorize, that content.

There is a tension here though between data and information (i.e. processed data). Even in the thread title you refer to "pertinent files," which presumes a judgement based on some analytical process about which file is pertinent or not.

Isn't there a danger of duplication (or multiplication) of effort, if after you have already reviewed the contents of a file, extracted some info in the form of a note, and linked to the file from the notes database, the file's contents turn up again in search results alongside the processed notes? It could introduce unnecessary noise into the process.

So playing devil's advocate, I'm arguing here for a tiered process, where lack of total integration is actually a good thing, and where one would use different specialised tools for both sides of the process, reserving linking for reference, in case you ever need to go back to check the original source.

But I realise there might be genuine need for total integration for some uses. Maybe Zoot can do such a thing?

jimspoon wrote:

Very good questions. One can imagine various degrees of integration of
"notes" and "file content". For me, the more integrated, the better.
So that when issues a query, the results may contain content from a
notes database, and content from files, mixed together and organized in
any number of different ways.

Well, a radical solution to this is for the integration to be at the note database infrastructure level, i.e. whereby the individual notes are themselves searchable files, such as plain text. This is Outwiker's take http://jenyay.net/Outwiker/English Keepnote also took a similar approach, but it seems no longer developed. Dokuwiki also holds its info in text files, and can be configured to run locally.

In all such cases, the desktop search programme will be able to fully index the notes along with any other files, and to provide integrated search results.

One more take: Archivarius is a very powerful indexing and search utility which can read a multitude of formats http://likasoft.com/document-search/index.shtml

This includes e-mail files, as well as some PIMs databases like Mybase.

So if you use a PIM supported by Archivarius you can concurrently search and get results from pertinent files, the information manager and email.

One complementary way to improve classifications across all repositories is to include tags _within_ the content. So a tag like "#project_X" could help the search engine fetch the relevant material wherever it may be.

Thanks for all the input! Dr. Andus you are right to play the devils advocate on this one.

There might be all kinds of smaller or bigger steps toward better integration of notes with external files.

One just popped into my head, and maybe it has been done somewhere. How about a links to folders, rather than just specific files ... so that when a file is placed in that folder, the info manager is aware of it, and will return "file" results in the search results. There wouldn't be a need to find the added files and drag them into the notes database.

Another possibility - linkage of notes database tags/folders with file system tags/folders.

I've been keeping an eye on Outwiker ... the performance isn't very fast, probably because it's written in Python. Jenyay has come out with a new version, based on an updated version of python I believe. Haven't tried it out yet.

jimspoon -

UltraRecall (from Kinook) does what I believe you are looking for.

- You can import a file system folder and specify whether you want the contents (e.g. PDF file) or just links to the folder content. see http://www.kinook.com/UltraRecall/Manual/importfolder.htm
- You can then use the Synchronize function (manual action) to keep the contents in UR in synch with the folder you import as described above. see http://www.kinook.com/UltraRecall/Manual/foldersynchronization.htm
- When you then perform a search, both internally stored and externally linked contents are included in the search results assuming that you have included the file extension as described at http://www.kinook.com/Forum/showthread.php?t=5078&highlight=search+files

I've found UR to be pretty strong in the search department and this might be another solution to consider....

jimspoon, hi there. I have been looking at Correlate K - Map, I have never used it, but it might be something to look at. If anybody knows more about this product, don't hesitate to comment.

I am not very active on the forum, but I felt I would just add a comment to an interesting product.

Note, they have just released a new version of their software and are looking at further developments.

good luck... jp

PS, I forgot to add the link to Correlate K - Map... here it is:

http://www.correlate.com/

jperlman wrote:

PS, I forgot to add the link to Correlate K - Map... here it is:

http://www.correlate.com/

Correlate has long intrigued me. It's been around for 14 years or more, though I gather from the website that it has changed hands recently. A lot of what Correlate can do, TheBrain can do as well, though Correlate provides a more standard outline structure.

They seem a little confused about how much it costs. On the main page it says a single-user license is $89, while in the FAQ it says $49.

I woud be interested in reading about your findings.

Steve Z.

I emailed Correlate about the price discrepancy:

Hi thanks for your heads up.

Correct price is $ 89 (net exclusive any local tax ) for a single lifelong license to Correlate version 8 for Windows. Error is due to new website design and content.

Each license is for a named user and his/her own devices ( max 3 ).

Best regards

Halvor Kalve
Correlate Team