CintaNotes, etc (meow!)

Posted by 22111
Nov 5, 2022 at 11:28 AM

Let’s put this straight.

What I mean by taxonomic coherence, you could also “pure trees” or “dimension delimitation”; in folder systems, you mix dimensions, simple example, you have countries, by some hierarchy (let’s call this the tree G for geographics), probably even with more or less artificial inter-titling (i.e. not Europe - France/Germany/GB/etc, but Europe - Scandinavia - then some, Europe - Former Eastern Block - then some, and so on), additional grouping which is not necessary but seems convenient; then you have political phenomena, again in a tree (let’s call it P for politics), e.g. corruption / so-called “elites”, justice, police (which are closely connected in fact, whilst conceptionally quite distinct, since the former, in theory, should derive from another “state power” than the latter; not even legally so in Germany though and, factually, in very few countries indeed: hence country - justice - police is not as aberrant as it seems at first look…), social politics, “Covid” policies, etc., etc.

Now you entangle trees G and P, by either putting P into G, or G into P.

(As an aside, trees need a source item, but it’s not necessary to display that source item, and if I remember well - I may be mistaken, MyInfo and/or RightNote don’t display it, whilst UR certainly does, and that’s perfectly unneeded and will cause about 1cm (or perhaps “only” 8 mm) horizontal space lost all over the tree, causing more or less automatic horizontal scrolling which is not “beautiful” at all.)

Then, in a folder system, you are stuck with that, except for manual rearrangement of it all, and then you’ll be stuck with that new one again; the same is true for trees with linking as in UR as described above: Upon my base, “physical” system by G, I superposed an additional, _mainly_ virtual system P, and, as said, _almost_ anything I do in there, in reality is done within the respective sub-folders in G.

Now, why do I say “mainly” and “almost”? Because by (current) necessity, the “virtual” tree P is not virtual in itself; I’ve got too many elements left which are assigned to that tree P, somewhere, but not (yet) to any of the countries / entries in tree G.

Thus, the main entries in P are real, not virtual, and then just the “country swaps” in there are virtual (i.e. UR links), in other words, in G, everything’s real = physical, whilst for example in P, the title “Police” is real, whilst its sub-titles “France”, “GB”, etc, are all links, to real G - France - Police, G - GB - Police, etc., whilst the entries P - Police - Other, P - Police - Not assigned, etc are real again; note that you can create real, and virtual, child (title, or any) items “below” real parent items only, but anything you create “below” a virtual parent item, will be in reality created “below” the linked parent, but also be visible, as another virtual sibling, between its virtual siblings, thus the possibility to process, any which way needed, the real data, even within the virtual representations.

Note that this also implies - and that’s highly welcome indeed! - that when working in real P data, the respective virtual, “country” P data is visible nearby (and not in some remote part of the global (super-) tree), so that, within P, it’s easy to move not yet correctly assigned data (real here) to those neighboring (here virtual) sub-titles, in fact = in real storing them within their physical environment to which, in this system (i.e. G = all physical, P = mostly virtual, except for “residuals”), they belong.

Also note that I could easily “straigthen out” my somewhat (in P) “hybrid” system into something perfectly neat in both trees (P and G), by just moving all the physical sub-titles from P to a newly to be created “country” “Non-assigned” within G, then link them again back to the respective P sub-titles. (Both in P and G, you could also, and for example, have two sub-titles, 0 for Inbox (meaning “non-assigned YET”, and Z for the above “non-assigned” / “Other”.)

Now I could have done it the other way round, having all the real data within P, and most or all of (see above for partly mix-up or neat separation) the virtual one in G, but the way I have done it, all country data is physically together (well, they are relational db tuples, so this is only true on the conceptual level, not on the technical one…), and in theory (i.e. if I could hold all my data in just one db (which would be correct for Postgres, but is impossible with SQLite), ALL my country data, not only the political one, would be together that way, and I’d prefer it that way, also, and not to the least, because some countries have got multiple “specifics” which don’t enter easily into a systematic taxonomy, applicable to all countries, and so there’s no need to create gratuitous headings and sub-categories in some systematic sub-tree, which only contain just one single country sub-category then; obviously, your needs might be different, depending on the nature of your db.
. .
Now for hierarchical tagging as seen in CN (CintaNotes). Coming from a folder system, you might be tempted to use the same tag tree for mixing up the two dimensions P and G as described above; obviously, you should not do this, even more so since what in folder systems with transclusion calls for some “work” as described above (and which is outright impossible in folder systems without transclusion, so such systems should be excluded from start on, and certainly not subscribed to (“Ulysses App”, anyone? or is there some way I might have overlooked?)), in hierarchical tag systems is really easy, quasi system-inherent: You will of course create P - measures - some measure, and assign that tag, and will create Countries - GB, and will assign that tag, too, instead of creating a mix of measures and geographic classifiers.

Unfortunately, there are two problems.

Problem one, specific to (all?) current tag systems: You always get the final intersection of your filters (whatever they are), and then that (in case very long) list of items is just ordered by some technical data (alphanumeric, creation or last access data, access frequency or whatever): You will not get some comprehensive, virtual tree, created by your filter compound, let alone with the suitable inter-titling - at least that’s the situation with CN if I haven’t overlooked that functionality.

As an aside, compare with Ultra Recall where you can “search” (i.e. filter to some extra pane) by several criteria (as explained by me above), and then sort by several criteria (i.e. within the results list, you click into the column header, then control-click for second sort criterion, then control-click again for the third one if needed - I think it’s limited to three but may be mistaken about that), and whilst you will not get inter-titling, you can choose the columns to be displayed, and their horizontal arrangement = visual “order”, and that display order can obviously by different from the above-mentioned sort order, but you can get some “somewhat replacement for inter-titling” by a smart combination of columns sort and columns display order; unfortunately, those “filter/display sets” are not storable in UR, so getting them back if you need to switch between some alternative ones, implies some “work”, each time.

On the other hand, with / in CN, this “smart ordering and sorting” does not seem to be possible at all (?), since to begin with, it seems - as implied by the then very limited sorting alternatives offered - that in CN, filtering by “P - Covid - some measure or some other measure” plus “G - GB or France” gives the same, i.e. “both systematically and geographically unordered”, result as “G - GB or France” plus “P - Covid- some measure or some other measure”, let alone switching the measures within…

It’s obvious now that such a tag system - IF I’m not mistaken about CN in this respect that is -, albeit being “hierarchical”, is less powerful, and by far, than a really smart folder-based system like Ultra Recall… but it’s evident, too, that CN could be enhanced accordingly, i.e. its current fail (again: IF I’m right here) is not system-inherent but just due to missing code.

The second problem lies in the fact that “dimensional coherence” is not systematically attainable. Whilst political measures vs. countries are clearly distinct dimensions, lots of taxonomies are more or less arbitrary; I had wanted to give examples from flora and fauna, but my respective knowledge is too poor to dare; on the other hand, we all “know”, i.e. have heard somewhere, that the usual botanical and zoological taxonomies are deemed arguable, in many respects, so…

So, whenever the delimiters are obvious, we should act accordingly, i.e. clearly distinguish, and when they are not, we have to live with it, but clearly distinct categories would be a prerequisite for pivoting automation.

I refer to my musings here about what I called a “tax(onomy)” column; in fact, most of the DBs we’re discussion here are relational ones, their hierarchy being realized by the so-called adjacency list model, and it’s as simple as it gets: For every item, its parent is stored, and then a recursive routine puts it all together (with additional info for sort order, etc., but e.g. the indentation level is determined by the lineage to be built, not upside-down but bottom-up - hence big scaling problems when your data set grows, and in the absence of enhancements, i.e. de-normalization = redundant data).

Thus, the db system does not distinguish between what you could call a “real folder item”, i.e. an item that really categorizes its child items, within your information system, and any other (parent) item; e.g. UR has got the additional column “has children yes/no”, but which already is (minor) redundant data, just to speed up the gathering / tree building; lots of such “parent items” have got just some “child items”, or even just a single one, and do not (sub-) categorize but just come with (one or more) “sidecar(s)”; on the other hand, a real category could just contain a single item yet, so just from “child count”, the system can not decide if some item is to be treated as a swappable folder, upon pivoting, or not.

As an aside, that phenomenon is the (again, not inherent, but just practical, i.e. due to their current realizations…) flaw of the so-called “Miller columns”: they don’t distinguish between category parents (where it’s indicated indeed to open up a “child pane” for what they contain), and “regular items”, just and additionally serving as “parent items” to, i.e. as containers for, some other, intimately connected stuff, but which for convenience reasons you want to put into some other, “child” item(s); examples are not only some longer press article, and then relevant reader comments within a child item, but also some scientific article, and then a reply to that, and perhaps even a counter-reply, or even some longer “conversation” between more people on that matter:

But it’s all just ONE item, category-wise, and to be categorized within the category/ies to which it belongs, and NO need whatsoever, from start on at least - this appreciation may indeed change (much) later or even never, and in case could and should be handled accordingly THEN -, but the idea of information management also implies the avoidance of a proliferation of “categories” of the same nature, and for example, completed legal cases should be “handled” in bulk, with, of course, the duplication of individual parts (sic!) of them into appropriate receiving contexts.

Above, I spoke about “formatting” the cloned (=real), and (i.e. in reality: i.e.!) the cloning (=virtual) folder entries; in fact, in UR, they both get a tiny, identical arrow in their icon (which is not very much visible, hence my additional formatting), and then my formatting is unfortunately identical, too, since, as already said, any action upon a cloned, or cloning item is in reality done upon the cloned item, so we would need an additional column in order to distinguish “cloned” and “clone”, by different formatting (I think Devonthink did that, but I also think they don’t allow for user-sided tree entry formatting…).

Thus, in UR, you can not easily distinguish between the two (in fact, there is a system attribute that can be displayed with a lot of fuss around it, in the “Item Attributes Pane”, not practical at all), so you have to distinguish by position, and yes, when your main categories in the db are, as in my “politics” db, P and G, that’s easy, once you only put “originals” into “G folders” (even replicated in P that is), except for some other, P-original folders (as described above, and, as said, not necessary when you move them into “G - Other/Unassigned (geographically that is)” instead); it’s obvious though that in a (in case much) more complicated system, the distinction between “physical position” and “clone” could become blurred, in some instances, so visual distinction between the two seems important indeed.

And then, with an additional column, not “has children y/n” but “is category y/n”, pivot automation becomes possible, i.e. the automated “creation” of the necessary sub-categories, in fact the cloning of the already existent ones, but according to a new subordination scheme, and obviously such schemes could be stored on their own, making the swap immediate. (With the prerequisite, obviously, that you would not maintain the above-mentioned mix but systematically create “Not assigned otherwise here” sub-categories within any categories, then create all your “content” items within the target main category you will have decided upon, in this example, within G, and then in “G - unassigned” in case, instead of creating* them within the P part if no other G sub-category applies. * = by “create” I don’t necessarily mean just the creation of the item, but the original assignment, by moving it out of some more general “inbox”)

So far for semi-automation, with the user creating those “swap alternatives” by hand, i.e. by working on a higher-up tree, just comprising all items declared as “is category folder” / “is taxonomy item” / whatever you call those ordinate elements, and possibly with some more semi-automation, by “standards” / “key- words” within those “tax items”, e.g. it would obviously not necessary that the user themself should do the swapping (even in bulk, as described for UR above, by - manually though - selecting multiple entries, then “swap” them into their different target positions by just one command), when the “dimension” is “countries”, but all “countries” “tax items” should bear some additional attribute “c”, and then the user would build the alternative tree in the form “some code - some other code - country - some yet other code - etc” - you cannot use “indentation level” for this btw, since that would ask for very artificial, empty levels in-between, in many parts; and of course, just one single column (for every item that is) would be sufficient, “tax”, with default = 0, or 1 = yes, or then some “helper code” (1 or more chars), which would imply the “1” again, obviously, but also would designate the “dimension” of that “tax item”, and the more your data then is “dimensionally coherent”, the more automatic those swaps will get, with very little manual user intervention on the occasion of the original alternative tree building and storing of that schema.

It also becomes obvious that in a more integrated db (instead of more or less needing, for somewhat 350,000 items, about 30 different SQLite DBs, as in my current case - since you simply can’t put them all into the same db, and then, the question arises how to distribute your stuff into the multiple ones, considering the then unavoidably raising combination needs between them…), i.e. in a db comprising all your stuff, you will want to do most of such swaps just within one big “area” of data, but then, too, in some cases, you will want to gather and rearrange data from distinct “areas”, just in such “alternative views”, and without affecting, let alone “destroying”, “lineage-wise”, data outside of your “pivot scope”.

Hence the absolute need for min-3-pane outlining, i.e. the current tree pane of 2-pane outliners being split into a “cat” pane, comprising the tree of all categorizing entries, and only those, and then one or even more than one intermediate panes, of which one contains the “contents” of the currently-active item in pane 1 (be that just a flat list or even some tree), and more intermediate panes might be subject to discussion indeed, e.g. some pane aggregating “dispersed” elements (cf. the intermediate pane in Ulysses App, and similar realizations); the moving of any element from pane 1 to pane 2 and vice versa being done by just assigning a “1” or a char code to its “cat” column, or then the deletion of such content, with automatic reverting to default “0”.

Obviously, I’ve done, with(in) UR, the “max you can get” with current(ly available, known) software, but it’s evident, too, that there’s room for big improvement, and the suggested enhancement to the basic adjacency list model, incl. the necessary code enhancements, is elementary.

At the end of the day, it’s all about separating form and content again, so as to be able to make the form flexible (content will automatically “follow”) without creating (unnecessary but when created, interfering) over-complexity, while preserving the advantages that have come indeed with most outliners’ mixing up the concepts of “folder” and “file” - ironically, those I know which don’t, don’t do it any better, currently, and there’s room for improving e.g. the NTFS, too…