Forum Posts Feed
Best of Both Worlds, Twice - I think this was the missing element to a perfect system: The "one db, one tree" nexus is dead.
< Next Topic | Back to topic list | Previous Topic >
Posted by 22111
Dec 12, 2013 at 12:51 AM
Sidestep first: I’ve said this before, but to no avail: Whilst MS Word formats level x of your outline this way, level y that way, right as you want to have those, NO outliner of my knowledge does so. Of course, this must not interfere with your own individual formatting, i.e. you should remain able to bold some entries, to underline or italicize others, but nethertheless, the outliner should have standard level formats which you enter as “styles” for those indentation levels, and this could be realized by color or background color (not only dark colors with white font (which would perhaps be ok for levels 1 and 2), but also, e.g., a bleu or chamois background for that line/entry/item if it’s level 3 or 4). Such a system would greatly enhance navigation within outliners like UR where you will end up with a 5- or 6-digit number of items within one single database.
Now first subject of the above, just as a reminder: “Mix up” your files, and your outliner entries; no file synching necessary; just links to folders, and then always-up-to-date run-time display of those files in your item list in your outliner, and access as any internal item, via multiple viewers within the same frame; see my post some weeks ago which developed this concept and also a three-frame display for large screens.
And now for again a “best of both worlds” concept which, as the above one, might revolutionize the outliner world - and now imagine those combined…
As you know, there are db-driven outliners, and text-based outliners, hex, xml, proprietary formats of all sorts. As you know, good search, and deep-linking (= links to individual items) only are available for the first kind.
Thus, the first kind seems to be preferable from this “immediate access to anything” pov (incl. easy updates where necessary, i.e. no additional db’s that monitor numerous single outliner files, for necessary updates within them) - ONE db.
On the other hand, such a concept bears the above-described problems, you’ll get (more or less) “lost” within your 5- or 6-figure number of items, let alone scalability within a corporation or such; the big-tree quickly becomes too big, let alone access rights M - who’s entitled to access what? And what about response times?
I should say that I never delved into db expertise; what I know about relational db’s, is just what about “everybody” knows about them, so it might be possible that what I’m saying here is not entirely new a concept, but even if it has been realized in a similar way, somewhere, it seems time to apply this concept to outlining… and then, perhaps, it’s as completely new as it currently is for me.
The “trick”, as I might say, lies in the breaking up that traditional “unit”, “one db, one big tree”, meaning, why not have one big db (for a start, and then, within a corporate network, this could be easily be scaled into clusters of db’s, with some inter-processing between the physical data (physical access) and the logical access sw.
Why not have one big db, then, but 1,000 or more independent trees (and then it becomes so easy to manage access rights, too), which all take their data from the one big db… but with (if needed/wanted) complete search access to “everything” (or to what in your department and/or in your corporate role will thus be available to you).
Since I envision a 3-frame gui anyway (but technically, a 2-tree gui would even suffice here), there will be no problem to have one, but virtual, tree, just for referencing sub-trees, and then there will be those sub-trees (and as indicated some weeks ago, either you will have two sub-trees in frames 2 and 3, or you will have just one such sub-tree in frame 2, but with an additional document from anywhere (that same subtree, external data, or any other item from anywhere - of course, beneath tree 2 there should be another list for quick navigation within such frame-3 content, both just for navigation and for linking to it).
Of course, there should be “interoperability” within two such independent trees within frames 2 and 3 (and both listed in frame 1) - copying, moving, linking/cloning; in a corporate environment, you also could have, e.g., “writing access” to one of these trees, and just “reading access” to another one, or any other form of access, i.e. “new-writing access”, but without “editing/deliting access”, and so on.
All this is very easy to do, technically, traditional outliners just do “too much”, by building up the “whole tree” for the entire db (entire file), instead of there being TWO such tree building-ups, one just for sub-trees (= levels 1 and 2 if you prefer to look at them this way), and then multiple such sub-trees of perhaps 20, 50, 120 items: just the kind of data that today, you stuck into the “intermediate” levels of your “big-tree”.
And as these independent sub-trees (but which can be, as said, extensively cross-linked, which is not the case for 1,000 text files or such) are quite shallow/flat (as said, most would not even have 120 items), their building-up would be made by run-time, just from ID lists (in fact, ID arrays, since there would also be needed indentation and formatting info); this way, updates/renames (= anything for which you need “clones”) would be very easy here, again to the contrary of a 1,000 xml files system.
Note that with a multi-db system as is possible with MI (since there is not only “search within the db” as in UR and such, but also “search within all loaded db’s” and “search within all db’s (within the working directory)”, search over spread data is quite ok, but the same is not true for inter-db-linking/cloning. You will also note that the concept of hoisting is not as needed as in big-data trees anymore.
Thus, this new paradigm preserves the advantages of the db-based outliner, but does away with the unmanageable big-tree outliner concept. And it’ll be scalable to any corporation size, whilst being perfect for one man/woman, too.
Posted by Pierre Paul Landry
Dec 12, 2013 at 02:23 AM
22111 wrote:
———————
so it might be possible that what I’m saying here is not entirely new a concept, but even if it has been realized in a similar way, somewhere, it seems time to apply this concept to outlining… and then, perhaps, it’s as completely new as it currently is for me.
The “trick”, as I might say, lies in the breaking up that traditional “unit”, “one db, one big tree”, meaning, why not have one big db (for a start, and then, within a corporate network, this could be easily be scaled into clusters of db’s, with some inter-processing between the physical data (physical access) and the logical access sw.
Why not have one big db, then, but 1,000 or more independent trees (and then it becomes so easy to manage access rights, too)
———————
I know of at least 2 applications that are designed from the ground up as you describe: Ecco Pro and my own InfoQube. And yes, IMHO, it is much easier to manage information using a number of “smaller” outlines than a single “huge” one !
Pierre Paul Landry
IQ Designer
http://www.infoqube.biz
Posted by 22111
Dec 21, 2013 at 12:36 PM
The above discussions mixes up two or three different concepts.
First, we have dedicated file managers / file commanders. Here, you chose the directory/folder in your file system (internal hdd or external devices), and will then get a list of what currently is in that directory / folder, those entities can be original files, or links / link files, e.g. .lnk files (XT); I don’t know if the most recent Windows links are necessarily files, too; technically, it would be possible to just have link entries in the file system which are not files on their own.
Variant: If you want to gather “project files”, i.e. various files from different locations/directories, you create links to them, and which you put then into a new directory named after your “project”, and which is a regular Windows folder which can, of course, contain such links / link files, and also original files; you can even rename the links (but not the original files, without breaking the link), or move/copy them to other folders.
Then, there is a variant, present in some of these file commanders, virtual folders. These file managers do not create .lnk files or such but create proprietary databases for files from any folder, from which then they display the db entries within a quite regular “lister”, as they would display the content of real folders, and their specificity is that you only see those “virtual folders” from within the file manager from within you will have created those “virtual folders” and their respective “content” (= internal, proprietary “links”/references to real files) - this is all rather basic, and I believe I remember that ordinarly, the entry in the “virtual folder” is not updated together with a rename/move of the original file, but this “breaks the link”; this might not be the case today in every such program, though.
In practice, this concept appears much less productive than it seems to be “on paper”; I tried with every such file commander (not only for some trial days: I own some 5 or 6 paid ones, and then some free ones, too), and it’s simply not worth it imo.
Then we have IMS’s, outliners or other, which “read” files into their tree, their db, or whatever. Here again, we usually do NOT have a precise real-time “image” of what’s in the file system, within a given folder, at any given moment, i.e. there is not, as is in dedicated file managers, an immediate reading of the file system entries, then display of these entries. There has been a little “discussion” in the Ultra Recall forum of you follow the link to the original thread within this recent UR thread: http://www.kinook.com/Forum/showthread.php?t=5219 .
But there again, you must differenciate between “just linking” which in UR’s case means creating a link ITEM for this (which is the reason it takes some moments), whilst in fact, you also could imagine just a “list entry” within the tree, and the creation of a link item only then if you need that for entering metadata for that file (the abolition of ADS metadata within NTFS is catastrophic in its consequences for everyone, but then, Mr. Balmer or Ballmer did not do anything intelligent in his entire MS life, except for his own income), so real-time updating this could be speeded-up.
As said before, UR has two ways to do this, just linking (but with creating a link item for any such link, hence it’s not that fast here), and another concept, importing the TEXT of that linked file (Word, pdf, some other formats) into its freshly created link item, if you set its setting for that import that way, and in this scenario, you can search the text of the “just-linked” doc from within UR, which in many cases is a very good thing - similar to TheBrain, where also some users use the IMS tool as a reference tool for thousands of pdf’s. It goes without saying that in THIS case, importing/reimporting/synching a (big enough) folder to UR “takes hours”, as has been said above (recreation of those parts within the UR index).
As I have said before, there would be a lot of synch work, and of maintenance for making this possible (renames/moves of linked files when UR isn’t running, e.g.) if you tried to synch files - it’s all very much easier when you just LINK to FOLDERS, BUT THEN DISPLAY THEIR CONTENT, this is not mutually exclusive, on the contrary - but such a concept implies that you do your “internal/further” linking within the file system, not within the IMS/outliner, i.e. an optimized UR would fetch the contents of your folder c:\abc every time you “go” to that “folder link item” in UR, and display all its content, incl. any .lnk files in that folder.
Now for what Stephen says about TheBrain - I do not doubt that all this is correct, but we are not informed how exactly TB “does it” (whilst I explained here how UR “does it”), so some further explanation on this would be very helpful.
As implied above (it becomes evident from the thread linked within the linked thread), UR does not do the synching automatically (and doesn’t monitor renames, let alone moves), but starts a manual synch by keyboard shortcut, and that’s my point:
Whenever an IMS is not able to monitor file system changes, any moment (e.g. by having a little add-on that’s running anytime, even when the IMS itself is not running, then synching between this and the IMS whenever the IMS starts to work), you will quickly end up with de-synch, so this should be avoided imo, cf. what I described some weeks ago - it’s perfectly possible to INTEGRATE a REAL file commander into your IMS, and that’s then quite another thing.
This being said, UR’s “manual synch” is not that bad, it’s so much better at least that most competitors in which there is no such synch function, but where you would have to update all your links by hand - my current system is to display the respective file folder, in any file commander of my choice, together with the respective outliner folder, i.e. a macro checks if I move between “siblings” and such, in which case no new folder display in the file manager is needed, or if I “change” the respective folder, and then my file commander displays two folders, side by side, which is not technically necessary, but which I did in order to see “how it all works best”, also in a possible corp environment:
In the left pane, I show “sibling” files, and current ToDo files, i.e. I apply two or more filters to a folder containing a 4-digit number of “standard” files, and in the right pane, I show “relevant” files, i.e. the file manager displays the respective sub-folder containing files “relevant here” (= original files or .lnk files, I always work within XP), which in my system is “external files”, files from the web, downloaded pdf’s, and all such (btw, when such a “deep” folder doesn’t exist, the macro displays the folder “1-up”, or, if that’s non-existent, too, an even broader, more “general” folder for that subject, and so on - you certainly get the interest of such a concept for a corporate environment), whilst in the left pane, it’s the “self-created files” for the same outliner entry/category.
I divided up those “files to consider here”, into two different folder systems, because I think that for “internal files” (= my left pane), you need much more security than for “external files”, so why not have them managed by the same macro/script/program routine, but from different physical sources (and as said, my separation into “one big folder, then filter” for “internals”, and “deep sub-folder file sub-system” for “externals” is only in order to experiment with both concepts; in the end, you could choose the first, or better, the second one, for both kinds of files); as said, there should be an “end product” where you would not have links for display in a file commander, but real-time display, file-commander-like, in a frame/list within your IMS itself, together with the immediate siblings and links within the outline body; entries from several sources, put together in a breeze, and divided up there by separator lines, e.g. - I call this “natural hoisting”, i.e. separation of the tree from your immediate working environment.
Again, it would be of high interest to know exactly how TB does linking/referencing, for single files, and for folders, respectively, in order to see where it currently stands between “regular outliners” and dedicated file commanders, and especially, where might be its possible advantages here over UR.
Posted by 22111
Dec 21, 2013 at 01:08 PM
Pierre Paul, after writing the above (first post), I expected some argument indeed, but coming from a possible ConnectedText advocate, since in CT, there is one big items body, but which is NOT constructed as an outliner, and from which then there can arise multiple outlines (as they say), so I expected the above then not as brand-new as I had presented it (without thinking of CT).
Now this argument comes, but from a different “corner”. I have to say I don’t know EccoPro at all, since my try to install it, some two years ago, just brought my system down: It’s very rare an application give my XT system a blue screen, but in about 10 or 12 years, the Ecco installation succeeded in this.
As for InfoQube, I don’t know it well, but from my short trial, I didn’t see it follows the above concept, I just saw the similarities with all those “big” outliners that all come from the opposite end of thinking, which is, “gather your stuff into a tree, and when that tree becomes too large, do hoisting, or break it up into several trees” (with MyInfo being the only (?) application which then permits global search over info within these different trees), and there seems to be no such application that currently is able to properly administer such a compound of different trees (which up to now, always means, different tree files).
If I am mistaken here, as far as IQ is concerned, I kindly invite you to describe its conceptual architecture, as I did here, these last weeks, for an IMS that would integrate such concepts and more but which doesn’t exist yet.
Perhaps IQ is far more advanced than I currently know and did expect, and this would be exceptionally brilliant, but fact is, nobody here ever mentioned such outstanding characteristics of your IMS here, so I felt save to not suspect them within it (and a similar consideration would apply to Zoot: there, too, I make assumptions of functionality not being there for the unique reason that it’s never been mentioned); so some detailed clarification would be more than helpful.
In just some word, my idea above (and which coming from quite another context, already has been realized, by CT, and probably by you) was that from what I see, outliner developers “all” (? so you do not? and that shows in the architecture of your IMS?) apply the idea that there would be a big tree for anything within the db, a tree of 100,000 items if there are 100,000 items in the db, whilst I say (and perhaps you said, before me) that those trees could be stored in one big db, but should be totally independent from each other, and in my concept, there would not even be the possibility, for the use, to combine them all into a bigger tree, whilst technically of course, this would be feasible, in spite of any possible recursion problems (you either create a monster tree, or better, you just follow “natural parentage”, not also “step parentage”, but of course, that will leave out some info that might be really important here and there.
But then, I just do not see the necessity to build up one big tree from it all, so I don’t see the necessity to resolve such recursion problems that only would arise then (= from a practical pov; from a purely technical pov, you could “manage” even lesser trees to be outrageously self-recursive).
Btw, I suppose that ordinary outliner developers do all this “child-driven” design in order to avoid recursion; it’s interesting that (from what I remember, I might be mistaken here) all of them who do “clones”, then do not distinguish the “original parent” (which should be replaceable, btw, by a new one) from any new “step parent” then. And all this means, in other words, they do the “big” tree, because in programming, that’s easier than having independent trees with deep interlinking, and from which then you would build new, virtual trees: Here, without real good programming, you could create broad chaos.
So, in light of what I say here and in the other thread, it would be of high interest to know what IQ really “does”, behind the scenes, and if it overcomes some traps ordinary outliner developers avoid by “holding it simple” - much too simple, for me, and if you see it similar to the way I see it, that would be a big step into a new direction from which much better end products could come to the user. If it’s already there, in IQ, you hid it well! ;-)
Posted by 22111
Dec 21, 2013 at 01:12 PM
“The above discussion[s] mixes up two or three different concepts.” referred to the original thread, of course:
http://www.outlinersoftware.com/topics/viewt/5223/5
in which the possible suitability of The Brain, and of Ultra Recall, was discussed, but without going into the necessary details, there. Sorry for not adjusting my post after shifting it here.