tldr: If you had to organize a large body of content spanning many different topics and projects for easy reference in the creation of NEW content, what would you use?
-----------------------

Hey friends,

For background, I previously posted this topic: https://www.outlinersoftware.com/topics/viewt/8387/0/software-request-open-source-personal-content-repository

In that topic I decided to go with Org-Mode. I still AM going to go with org-mode as my main content CREATION tool (I'm currently spinning up my dream editor in Emacs), but I'm realizing that it's quite challenging to use as a reference tool.

It's simply too cumbersome to link and search files to create a truly useful web of connected knowledge. Yes you CAN do it, but it's seeming to be far more clicks and configuration than it's worth.

Let's say over the course of the next few years I'll produce hundreds of pieces of content (newsletters, outlines for videos, social media campaigns, etc.) many of them will be nested inside certain series, connected to various topics, attached to various projects. What would you recommend to STORE the content after I've created it?

The main goal would be to reference this content to build new connections and insights that would lead to the creation of even more content.

As a windows user, right now my best options seem to be:
- Evernote
- TheBrain
- A personal wiki

I'm already quite familiar with Evernote, and it would probably get the job done as a pure reference tool. However, it lacks the features for connectivity and discovery such as custom fields and backlinking.

This has me considering TheBrain or a personal wiki. The nice thing about a wiki is that it could be built with free and open software. The nice thing about TheBrain is that it seems pretty much designed for my purposes here and might even be able to replace my evernote usage.

I don't have much experience with either of these options and so wanted to hear your thoughts, particularly from those of you who have experience with either. Of course, if there's other options I haven't considered, I'm all ears.

When that is my goal I use TheBrain. TheBrain excels at emergent structure -- a topic worth exploring as you prepare your approach to this.

A personal wiki might eventually surpass TheBrain's usefulness for you. BUT with a personal wiki you are going to end up doing a lot of fiddling around the edges to get the wiki up and running the way you want, so that you might spend as much or more time taking care of the infrastructure as working on your core tasks of building connections and insights.

Evernote I would never use for this purpose. It's a bunch of buckets with no visual connection between the notes. Search in Evernote sucks. Not the right tool, in my opinion.

thouqht wrote:

The main goal would be to reference this content to build new
connections and insights that would lead to the creation of even more
content.

As a windows user, right now my best options seem to be:
- Evernote
- TheBrain
- A personal wiki

Paul Korm wrote:

When that is my goal I use TheBrain. TheBrain excels at emergent
structure -- a topic worth exploring as you prepare your approach to
this.

A personal wiki might eventually surpass TheBrain's usefulness for you.
BUT with a personal wiki you are going to end up doing a lot of fiddling
around the edges to get the wiki up and running the way you want, so
that you might spend as much or more time taking care of the
infrastructure as working on your core tasks of building connections and
insights.

Thanks Paul, that's useful. I definitely want to avoid having the management of my knowledge base become a major time sink. Setup, learning, and general maintenance is expected, but some systems just demand a bit too much. If you don't mind me asking, how long have you used TheBrain and how satisfied with it are you?

I've also just come across Zotero which seems quite interesting and powerful in it's own right... I know it's used primarily by academics for research, but it looks like it might have some useful capabilities.

I've used TheBrain on Windows and OS X/macOS for 13 years. Yes, very satisfied.

thouqht wrote:

If you don't mind me asking, how long have you
used TheBrain and how satisfied with it are you?

The learning curve for TheBrain is way lower than it might seem just from looking at it. Also, the developer has so many instructional videos that it makes it a cinch to get the hang of it.

Steve Z.

Steve's got a good point. And every Friday they host a 101 webinar with an interactive session for Q&A. They've been doing this for years -- a commitment that's pretty rare.

https://www.thebrain.com/support/thebrain-101

+1 for TheBrain. The newly introduced BrainBox has loads of potential for queuing up items for TheBrain. A possible downside is TheBrain's search. It's now an OS-referenced search. I think they at one point used a version of dtSearch (at least for Windows)---- I'd love to see them integrate Carrot2 into their search.

If you're looking at Zotero, you may also want to consider Citavi, though it's certainly geared more toward biblio-based curation, it's usefulness goes beyond that.

What is BrainBox? I did a quick web search but couldn't seem to find it.

--Chris

mdlynam wrote:

+1 for TheBrain. The newly introduced BrainBox has loads of potential
for queuing up items for TheBrain.

It's a component of their beta version (Brain 10). More information at:
https://www.thebrain.com/products/thebrain/thebrain10


Chris Thompson wrote:

What is BrainBox? I did a quick web search but couldn't seem to find it.

--Chris

mdlynam wrote:
>+1 for TheBrain. The newly introduced BrainBox has loads of potential
>for queuing up items for TheBrain.

Thanks for the link. I'm a long-time TheBrain user and this may be the release that has me most excited to get my hands on.



mdlynam wrote:

It's a component of their beta version (Brain 10). More information at:
https://www.thebrain.com/products/thebrain/thebrain10


Chris Thompson wrote:
What is BrainBox? I did a quick web search but couldn't seem to find it.
>
>--Chris
>
>mdlynam wrote:
>>+1 for TheBrain. The newly introduced BrainBox has loads of potential
>>for queuing up items for TheBrain.

+1 for TheBrain

Not only do you have the spatial connection of "thoughts" (jump link, parent, child) but you can also link directly to other thoughts from within the note attached to a thought. Add to that nested Types and nested Tags, and the information connection density is beyond excellent.

While Wiki's and even Evernote can be made to help with (re)discovery, I'm always amazed at how I find new connections in my brain. A central though is connected to a parent; I see the other thoughts connected to that parent, and off I go as I spot something there.

Although I'm an avid TheBrain user, I'm starting to look into two alternative possibilities.

Both methods are somewhat hacky, and during the initial stages I expect I will at some point feel like I'm sinking too much time into tools and not enough time actually producing stuff, but I suspect in the long run the flexibility it offers could be worth it.

The first is to keep all my notes etc in an existing programming language, say javascript, and piggyback on the tools offered by IDEs. Just to mention one example, one thing you can do by way of simulating a wiki, is to put the content of a wiki article in the body of a 'function', which will be given a name. Then this function will be referenced ('called') elsewhere, and typical IDEs offer Intellisense features that will allow you to jump from function calls to function definitions, and from funcion definitions to all the places where it's called/referenced. Boom! You have cross-linking/backlinking for free. (And of course function bodies needn't just contain plain text, but can contain links (function calls) to other functions as well)

There's a lot to be done to fully flesh out this idea, of course. (For example, you may want to make a lot of changes to the syntax files, so e.g. the notes you write in the function body don't get flagged as illegal syntax. But then, since we're not really going to run/compile this thing, it probably doesn't matter)

Another idea I'm very interested in is just to keep all my notes in html, but with custom semantic tags, and a dose of javascript and css for customizing and especially quickly switching between different ways of presenting and visualizing the data. The html can be handcoded (with input tools like emmet to speed things up) or preferably with a good wysiwyg html editor. Wiki functionalities will be achieved via profuse use of regex.

Both approaches rely a lot on search, and I initially resisted how disorganized this approach. But what I gradually realized during my use of TheBrain is that I spent a lot of time organizing even though the organizing itself rarely helps in terms of retrieval - I can always find what I want by recalling some key words and phrases (in fact, I often welcome the pleasant surprises that show up in my search results). All the different options that come with tags, types, and even parent-child relationships start to become more of a time sink, so nowadays I try as much as possible to enforce a "no type, no tags, all connections must be of the jump type" rule. Paradoxical though it may sound I feel the freest when I deprive myself of a certain sort of freedom

Above: "I think they at one point used a version of dtSearch (at least for Windows)" - this is probably being meant as a quality assertion, while in fact,

dtSearch either is an overpriced piece of crap,

or they very successfully hide its strengths, while obviously not being interested in individual customers. I trialed it again some weeks ago. It indexed "known" file formats, was not able to index file formats "unknown" to it, not even when they were more or less identical to .rtf but not totally identical to .rtf.

I searched for a way to "tell" dtSearch to "grasp" it, more or less; I would have been willing to accept some limitations, like I encounter in FileLocator (not indexing, so searches often taking 20 minutes on a modern and fast pc), where for example I have to search for accented characters not by "é" e.g. but by the corresponding 4-char rtf code (I have macros for the transposition): no way.

In normal circumstances, the above (non-) result would not have been surprising - it's perfectly similar to e.g. "X1" 's and others' inability to "find" something in any file format they don't index by default, BUT dtSearch is touted all over the web - and that tries to justify its price, a multiple of e.g. "X1" - as a "forensic" tool, so it's VERY surprising that it should not be able to search even just for text within files coming with a quite simple ASCII format like similar to .rtf, formats that you can read in any text editor of your choice - nothing "binary" here.

Of course, I looked up the whole manual for some solution for this miss; of course I thought it must be "somewhere", considering its alleged "forensic" character: nothing.


Then only I kindly contacted their support; I NEVER got the slightest REPLY.


Btw, for about 15,000 bucks or so, when I last time looked it up, you can integrate the then current version of dtSearch into your own software, BUT that software of yours must store its data in a NON-standard format (yes, that's VERY ironic, considering the problems described above), and then, your integrated dtSearch will ONLY search that NON-standard-formatted data of yours, NOT also any standard-formatted user data, and in practical terms this means that if you write some db-backed outliner, your integrated dtSearch will be able to search within that db, but NOT also search any .txt, .rtf., .docx, .pdf or similar file the user will have linked to within their/your outliner.

Of course, you can do it like UR does it, which replicates the text of external, linked-to files within the db, in order for UR then to do the search upon the text replica within the db, by internal means of the SQLite engine, and this comes with a lot of problems but doesn't cost UR 15,000 bucks or some... for the same result.

In other words, developers would have to be nuts to buy dtSearch for their software, in all use cases I can imagine, and as for the alleged "forensic" abilities of dtSearch, I didn't find them, even scrutinizing the manual, and they wouldn't tell me.

You know, some products (all over the place, not just software) are just marketed by superior pricing, and then the fanboys come along and scream, oh it's so good! Just recently, Apple's newest "smartphones", about 1,700 euro (around 2,000 bucks) apiece, probably even more with VAT higher than 20 p.c. in some countries, and they don't even give head.

Is crude language really necessary here to get one's point across?

Interesting thoughts, SmallDog.

Sounds like a job for plain txt files? Some markdown, some # and @ references/tags maybe?

As for organizing in TheBrain -- although I often know what I'm searching for, so that I can use the search, at times I don't. At such a time approximating my data points helps a lot. Broad categories, connections. I usually put info in one place. Once I've resurfaced it in another way, I may add more connections. This way the time investment isn't up front and is based on my actual use.

I think The Brain's best news is they've retired Java. Oracle is pretty silly in the licensing department, and I'm ready to agree with them Java should be priced beyond the reach of the sweaty masses, of which I'm a proud member.

It's hard to imagine a better tool for evolving knowledge. Gather facts, but, even better, gather relationships.

These days I use DevonTHINK where I would have used The Brain. DT is very plain looking by comparison, but it does a good job. My one wish, at least at the moment, would be more agile navigation in DT.

You can reveal a tag to see all documents with a tag. Then, to see the primary group for a document, you use command-R to reveal the group. I know, picky, but all in one navigation, drilling through associations by tag and group, would be nice. I'd lose a little of my Brain envy, because nothing does navigation like The Brain. Nothing.

At least nothing I'm aware of.