In my personal ongoing saga for the ideal reading/research note database, and based on the good advice of many on here, particularly Daly, i have come to find that InfoHandler and ADM 3 seem to hold the most potential (and, perhaps, InfoSelect, but i don't know if i'm willing to plunk down $250 just to try software).

However, one thing that has gotten me thinking, is what will happen to years and years of my research data, if i place all my notes and data in a piece of software that is proprietary, and one day the company up and disappears. It seems many of these note PIM's and outlining programs - such a ADM, InfoHandler, Zoot, and Brainstorm - are basically one-man (or, in the case of Brainstorm, two-man) shows.

As such, when the developer gets bored, moves on to something else, passes away prematurely, etc. etc., there goes all support, and as such, *future portability* of the information into next generation software and operating systems.

Thus, the proprietary format of IH and ADM especially (and other textual databases) frightens me, though, as a layman end-user of software, i don't know how much i should rightly be frightened.

Also, the lack of ability for one to use desktop or universal searching through these databases, such as that provided by dekstop search engines like DTsearch and others, also seems to be a bit of a long-term worry, as convergence and interoperability increase over time, and as hard drives get bigger and bigger, and more and more complicated with different types of file types, extensions, and formats. Why shouldn't we be able to use independent search engines such as DTsearch that have a degree of fuzzy, Boolean or Artifical Intelligence (AI) search capabilities, over a huge database of notes, thoughts, writings, and articles? Isn't all that prevents this the proprietary formats of the software?

In order to preserve one's hard work and years of writings and notes, what is most important to look for: that a piece of software save itself in xml? that it be possible to export to xml? that it save it rich text file (.rtf), or normal text file (.txt)? What makes one any safer than the other, in terms of long-term backup and carrying over into future os's (and perhaps even a switch to another OS, such as Mac OSX, etc.)?

Software developers reading this: is it not possible (and i ask this sincerely, not rhetorically) for you to use open standards saving formats such as xml instead of binary proprietary modes?

Sorry for the meandering nature of this post, but i am trying to make up my mind - once and for all - on which software i want to use for long-term note-taking and note storage, such as i'll need in my academic research. I'm half tempted to just switch to Mac OSX, since i've heard and seen so many beautiful things in the world of DevonThink (especially with its Artificial Intelligence recommendation of related notes), and the myriad outliners available there. Then again, i've already invested so much time, money and effort into Windows based applications over the years, that i fear that my desire to jump ship is not just a mutation of what some here call "CRIMP". :-)

Anyhow...any comments on the proprietary lock in issue, and how best to get around it, or what software is already looking ahead and using open standards such as xml, would be greatly appreciated.

Talal

I think that minimum requirement is a possibility of export data to some popular formats. Here I would like to mention my favourite netXtract again. Its Knowledge Base (collection of your notes with categories, keywords etc) can be exported to HTML (works best for me), XML or MDB (MS Access).

Wojciech

Anyhow...any comments on the proprietary lock in issue, and how best to get around it, or what software is already looking ahead and using open standards such as xml, would be greatly appreciated.

Talal,

That is certainly a good issue to consider. One thing to keep in mind is that even defunct programs, if they are stable, can run for years, as is attested to by those on this forum who still use GrandView regularly, and the numbers of people using Ecco Pro. Of course, it is important to keep in mind if these programs will continue to run on the next generation Windows operating system.

That's all the wisdom I have on this subject. But I guess I'm saying that if you really like a certain program, that may be more important than other considerations.

Steve Z.

With respect to global search of your files, another problem with many of the latest versions of PIMs, etc. is that they compress their files, so a global search engine can't read them.

Software developers reading this: is it not possible (and i ask this sincerely, not rhetorically) for you to use open standards saving formats such as xml instead of binary proprietary modes?

Talal,
Please see my reply: http://www.outliners.com/discuss/msgReader$4549?mode=topic where I mention that Surfulater does use xml as its native storage format.

Propriety databases and lock-in are a real problem for many people for the reasons you state. You also have to consider how you can get the information from one product to another, if and when you need to move on.

Talal -

The issues you raise are very important ones, particularly when one is considering how to assemble one's ideal toolkit. On the other hand...

Data portability can be a problem whether or not a reasonably stable future for a piece of software can be predicted. To take an example from genealogy, a powerful and well-established program such as The Master Genealogist may lock one into using it simply because it uses proprietory forms of data such as "witnesses". The much-reviled but essential Gedcom (GEnealogical Data COMmunication standard) format for sharing data amongst genealogists, however, offers no easy way of handling such non-standard data. This often makes it impracticable for those with large databases containing huge numbers of witnesses to various events to move to new software.

Where software simply manipulates text (much of software discussed in this forum), the dangers of getting locked in are usually much smaller. Programs like Zoot, Grandview, askSam, etc., provide a range of standard exporting formats, and even ones like ContentSaver (v2, that is) offer bulk ways of exporting saved web-pages from its propietory database in *.htm standard format. The issue is usually more one of time and inconvenience than of flat impossibility.

It is also often relatively easy to query one's data using decent indexed search programs (Wilbur, dtSearch and some others), even where the file format is proprietory, so long as the text is not compressed or encrypted in some way.

The problem, where there is one, usually lies with how the results of such searches are displayed. Ideally, one wants to see the results displayed in context, preferably in a format as close to that of the original file containing the research "hits" as possible (i.e., as if looking at the original Word file, Excel file, or Outlook email, or whatever).

Unfortunately, many Windows search programs can only provide this feature for a limited range of file formats (such the Microsoft ones above). For example, since the demise of Lotus Magellan (DOS), no modern indexed search programs have the required viewers to display search results from Grandview (DOS outliner) and Lotus Agenda (DOS free-form database) in their native formats.

Worse still, some modern indexed search programs cannot even index and search such files at all, even though the bulk of the file is made up of plain text - and this limitation may not only apply to discontinued file formats, but also to newer ones, such as those of Zoot and askSam.

For many people these are pretty arcane problems, and I wouldn't want to over-emphasize them. I have long since exported my Agenda files to Zoot (using Zoot's splendid importing facilities; Information Handler can also import them, too); and I know that I can relatively easily swap outlining files between Grandview and Brainstorm, or Inspiration, so I am not locked into GV.

I don't expect any time soon to see Zoot or askSam files properly displayed by search programs. But I am happy (using dtSearch or Wilbur) to be able to search these files for text strings, along with all the other indexed file formats on my hard disk. It is often more important to know that one of my Zoot databases contains essential information than to know exactly where in the file the information is. For that task, I can always open the file in Zoot itself and search again.

Hope that helps.

Derek

Derek,
You raise some good and valid points however there is one issue regarding lock-in and proprietary data I'd like to raise. That is the need for other applications to access information in these programs, without the necessity of an Export process. Programs that use open standard formats like XML enable that direct access. This is an important requirement for some folks.

Steve,

"Here is a less egregious, but nevertheless aggravating and time consuming, variant of "lock in." I am speaking, in particular, of Maple Pro (www.crystaloffice.com), although other programs may have had the same problems."


In my opinion there is no acceptable reason for making conversions as difficult as you've outlined. It is a great way to alienate your customers and in your case loose them.

We've done four updates to the Surfulater Knowledge Base structure in the past 12 months. New fields were added, internal representation of dates changed, new article templates added etc. and each time this has been totally automatic and invisible to the user, as it should be.

There are several reason why we've been able to do this. First and foremost we knew from the outset that it wasn't possible for us to get things right from day one so we designed a database update process into the core of Surfulater. Second using XML makes certain changes very easy, hence the name "eXtensible" Markup Language.

XML has been termed the ASCII of the future, which is a nice simple way to look at it. As previously said it is human readable and editable in any plain old text editor. But at the same time it allows for rich structuring and manipulation of information.

XML may morph into something else in the future, but you will always know you can access your information as long as it continues to use plain ASCII text.

I've been writing software for over 20 years and seen all the hype come and go more times than I care to remember, but I have to say that IMO the hype over XML is justified, and not to be underestimated.

Derek,

"As for enabling direct access to Zoot's data without exporting it, this would entail substituting XML for the current *.zot database file format. Is that what you have in mind? If so, how feasible would that be?"

I can't really comment on Zoot. I did spend quite some time looking at all the the various PIM, KM, Outline type applications early in 2004, including Zoot but my memory of it is quite hazy.

I'd certainly be interested in feedback from the folks here on Surfulater's suitability as an outlining program and would be pleased to take your suggestions on board.