Search and Grouping/Combining

Posted by 22111
Jun 18, 2022 at 09:54 AM

Once - versions 5 and 6 - MyInfo used some “not-standard” db, I’ve never known which one. For me, crashes were not rare, both in 5 and 6. None of those “outliners” I’ve encountered, remained “stable” beyond a certain number of (simple, just formatted text, no pics/graphics) items = db records.

Thus, not only above that number (range), but much earlier, “you” had to distribute your stuff into SEVERAL such db’s; “much earlier” since WHEN you have to distribute your stuff, you’ll have to do it in a sensible way (and problems arise when you would have wanted to hold together / link some “un-related” data blocks, unrelated taxonomy-wise, but NEEDED together… and worse, one of the two (or more) being needed in MORE than one (taxonomy-unrelated) context / area, e.g. financial things together with several businesses instead of just one, or sales psychology within the same scenario of several businesses; a lot more examples come to mind, among them legal things, where some matters would concern any of your businesses, some others just specific businesses.

Thus, given the “reasonable capacity” problems of our 1-or-some-user “outlining”!) desktop db’s (server-client is another beast, but excluded for most for pricing considerations (e.g. MS’s light version would not help), except for Postgres (but I don’t know any commercialized example for “our means” with the latter), we’ve got to “combine” several db’s, in whatever style:

You could either combine hundreds of db’s, even atomizing the above matters, than combine them in multiple ways, or you just do several db’s, “legal”, “financial”, “biz1”..., accepting that your db combinations then will also bring data irrelevant to the specific situation.

Now, with its ancient db unknown to me, MI was able to search over several db’s - if I remember well, that was over all OPEN db’s only, but I might be wrong here; also, search was quite slow but I might be wrong here, again. (No index, or some aggregate index, with quite complicated management, causing delays?)

When you’ve got your stuff online, on rented space, “they” (hopefully) will do the necessary to hold together all of your data within the same db, so you can search wherever you want (scope), similar to an office with their own server on premises (Postgres, MS, Oracle, MySQL (= Oracle again) but pricing of the last 3 is demential…).

Thus, whatever its speed (which was not atrocious, just not “immediate” if I’m not wrong as said), MI’s search functionality beyond your “current” db was highly welcome; the same is NOT true with today’s SQLite db’s if I’m not wrong (again): Neither UR (here I’m positive) nor Rightnote nor Mybase seem to offer search beyond your “active” db, why? Simply because for search, they rely (again, UR does) on SQLite’s built-in full text search functionality, and that’s (up to now) db-specific.

Now that MI, for its verson 7, also has switched to the (here and on “smartphones”) “standard” SQLite db, this might be beneficial for its stability (?), but probably, also its search beyond your current db might have been gone, and might not come back too soon?

And at the end of the day, that had been MI’s USP… - together with always visible additional data columns you might say, but as I have said here before, all the time, and then even with a some “mature” 6 version (trial), I “succeeded” (i.e. against my wishes) in crashing the tool within minutes, several times, and with just some records and just some standard data (minimalist trial situation); I admit that now with SQLite, these problems might having been resolved though.

As implied by me in another thread, it seems that the Ulysses developers use multiple xml files, one for each item, and then more files of whatever format for the management of those, so they might have written their own search functionality, for your “local” stuff, or then, some Mac db(‘s indexing / search functionality) unknown to me; undeniably, that is highly welcome and much better than what we currently get in Windows’ SQLite world; also, Ul shows the (first, or perhaps even any, occurrence of the) context of the search term in every item, and that’s way better than I’m accustomed to, so yes, search is Ul’s very strong point.

(I don’t know about Ul’s stability with high numbers of items (“pages”) though; its alleged data storage format should remain perfectly stable then, but the situation is much less evident about its data management functionalities, incl. index / search…)

Using an external search tool isn’t but a makeshift, and still: most of them are, inacceptably, by subscription now; after having found what you search for, you will have then to look it up again in your “outliner”; and after all, I currently don’t know any search tool which finds text (or other data) - even the human-readable full text search index text in UR - in an SQLite db - it seems the search tool developers all (?) assume “SQLite comes with its own search functionality”... but then, that’s db-specific, whilst search tools’ biz is to be NOT file-specific, precisely…

Thus, the chance to hold “it all” together, without stability issues, and thus to avoid, among other, search scope problems, is the big “argument” for web storage (where availability and speed depend on your web connection, obviously, and with even West Europe now fast-pacedly becoming a Third World area now…)... but then, whilst I think it would be known quite soon when MS tried to copy your local (desktop or “laptop”) data over to their servers, any web data leakage is almost never known but when they then try to sell it on open markets… and when they move or delete, instead of “just” copying…

It would be of interest to know about the relevant Mac “outliners”, well, of Devonthink, there’s not much left after all: What stability / navigational and search speed with half a million items within the same db (but they buried their 3-pane design, it being allegedly “too complicated” for the “regular user” - whoever that person may be who first pays for Devonthink, and then is allegedly unable to understand what Ul users seem to grasp easily?), etc., underlying db? (Filemaker?).

There are lots of DT and Ul users out there, but they never speak or write of these core characteristics of any really good information tool, and whilst Ul is marketed mainly to “writers”, DT is viewed as a “data repository”, before anything else, so that should deliver?

(With UR, you should “stop” around 50,000 items or earlier (and with pics in text, much much earlier…), much much earlier, too, with MI (before 7 which I don’t know, and from my own experience) and with MB (there again, old versions, from the trials of JP Miller)...

At the end of the day, both problems, that you have to distribute your data, instead of holding it together, and that you then can’t but search tiny fractions of your data each time, are due to the fact the the Windows “standard” SQLite simply isn’t good enough, so if on the Mac side that’s really better… IF, I said…

And of course, within a single data set, you decollate and recollate at your wish, building a perfect ontology out of your partial taxonomies, thanks to their multiple occurrences, everywhere where needed, thanks to transclusion.

Well, not in Ul(ysses, mind you): They “hold it simple”, I suppose, for their users not being overstrained again… “after all, they’ll already have to cope with the 3-pane U(ser)I(nterface, mind you), the poor chumps!” - as obviously the developers think… and yes, as the “user”, you can just leave menu entries alone if what they hide isn’t of current interest to you, so the “hold it simple” paradigm seems to be just a tiny bit dishonest to me… but that’s just me then.