Forum Posts Feed
Software Request: Open Source Personal Content Repository
< Next Topic | Back to topic list | Previous Topic >
Posted by nathanb
Sep 12, 2018 at 07:24 PM
thouqht wrote:
>
>nathanb wrote:
>Based on your finalists, it looks like you are looking for a front-end
>>index to a bunch of mixed media files (photos, vids, pdfs etc)? That
>>you are planning on storing the media in a file system archive, not
>>‘inside’ the database of the index system?
>>
>>Tagspaces is probably what I’d use if I wanted a ‘future proof’ index
>of
>>a file collection. It also does light note-taking as a front-end to
>>plain-text markdown files.
>
>Hmmm that’s an interesting thought… I think as purely a front end to
>my file system that would work great. However as you suggest this, it
>makes me realize that I’d like to be able to create links *between*
>files as well. This is something that tiddlywiki, org-mode, or dokuwiki
>do quite well. However, perhaps a sophisticated enough tagging system
>would accomplish similar results…
I feel your pain. Every time I get excited about links (especially to files), I immediately run into the inherent fragility of one-way linking, especially between different systems. It’s dependent on the names and locations staying the same. I think this problem is particularly annoying for file directories since you only get ONE hierarchy scheme and you can’t re-arrange it if your future self questions now self’s opinions on how stuff should be organized.
After years of CRIMPing and coming up short with a robust system that handles inevitable evolution of organization concepts (and a carousel of trying out different systems), the ONE method I’ve tried that protects against link fragility is the humble Zettel Code. Well, I call it my Zettel code, but I don’t actually follow the Zellkasten system. I just like the concept of a unique text string AS the link.
So now, when I’m working within my task manager, notes, reports etc and want to reference a file. I just generate a new Z-code, which for me is a timestamp to ensure that it’s unique unless I live until September 12th, 2119, in which case I hope we’ve solved linking. So the code looks like ‘Z18090121351. It’s auto-generated via hotkey and I use military time to ensure uniqueness to this particular minute. The timestamp-as-information is only coincidental as I frequently am linking ‘old’ stuff. It might tell me what order I ordered stuff in but has no actual bearing on the timeliness of the content it is flagging. I copy this code into the end of the filename/foldername/whatever-content I’m linking to.
Now I only have to follow ONE cardinal rule within my hodgepodge of a digital nightmare of ever changing concepts and rules. That I NEVER change any z-code I run across and I ALWAYS accompany every intra-system reference with a z-code. Beyond that, I’m free to rename/re-arrange as much as I want because I’ll always be able to find that reference via basic search. This stupid simple method has made all the difference and freed me from trying to establish THE perfect system and instead slowly evolve a combination of loosely connected parts.
It’s pretty cool to be able to reference my 2018 tax return pdf from within my notes, task manager, simple documents, calendar, email etc and that ‘link’ will still work under all the following circumstances:
-I ‘migrate’ all my files from OneDrive to dropbox/gdocs/NAS etc.
-I completely re-arrange all my personal files, both names (except the z-code part) and folder structure
-I change to a different note system/wiki/task manager etc.
The worst case scenario is having to search more than two places for the reference. The other downside is that all my systems and directories and names are uglied-up by these z-codes now. So aesthetically it’s annoying but it is a constant reminder of how robust my links are AND it’s a good flag that something is important enough to have been referenced from another thing.
This method has been the ONE constant in my digital life for about two years now and I continue to find new uses for it. I started doing this specifically for reference documents and now it is my go-to solution for a lot more. Then I started using it to flag tasks and projects. I now have my own subcategories of z-codes. ZM*** is a meeting and therefore a hint to search my calendars first for it. ZT*** is a task so I’d look first inside my task manager for the matching code. ZP*** means project which means it probably has it’s own folder in one of 3 different file systems and probably it’s own section in OneNote/Notion and maybe even it’s own Outlook folder if it’s for work. As I get more granular with it, I’m now to the point where I’m approaching true Zettelkasten territory, where each z-code represents a much more granular bit of info like idea, goal, category, quote, book etc… but honestly I’m not about to give up on more appropriate systems for those things by applying a silly plain text solution to all of it. I’m not ready to admit CRIMPing defeat and revert to just text files. But these Z-codes have been a super reliable life-raft that lets me play around with different components without interrupting my workflow.
Posted by Franz Grieser
Sep 12, 2018 at 08:37 PM
Thanks nathanb for the explanation.
What I am not sure I really understand: You have a file named (let’s say) “outline for sci-fi novel.doc” and reference this file in 3 other places. Then you’d have to add 3 times a code like “Z18090121351” to the file name. Right? So, depending on the file system you’re using, there is a limit to the number of references you can add to the file name. In Windows a file name including the folder path may not be longer than 255 digits (ok, there seems to be a registry hack that permits longer names but I haven’t tested that). What do you do when you reach this limit?
Posted by nathanb
Sep 12, 2018 at 09:43 PM
Franz Grieser wrote:
Thanks nathanb for the explanation.
>
>What I am not sure I really understand: You have a file named (let’s
>say) “outline for sci-fi novel.doc” and reference this file in 3 other
>places. Then you’d have to add 3 times a code like “Z18090121351” to the
>file name. Right? So, depending on the file system you’re using, there
>is a limit to the number of references you can add to the file name. In
>Windows a file name including the folder path may not be longer than 255
>digits (ok, there seems to be a registry hack that permits longer names
>but I haven’t tested that). What do you do when you reach this limit?
Sorry I wasn’t clear. I don’t have any filenames with more than one z-code. Maybe it’d be more accurate to say that the z-code is a special tracker for that file instead of the code representing a particular link between two things. In the case of only one note referring to only one file, the distinction is unnecessary.
So if I have several notes/projects/tasks etc referring to the same file, it’s going to be with the same Z-code they all share.
You bring up a pretty good point though about the difference between two-way linking (otherwise called directed graph, transclusion, back-links etc) and simple one-way ‘blind’ linking. I’m forever trying to figure out the best systems with true two-way links and we can only ever achieve that within a single database/wiki/app because it requires that the link itself (the relationship) be it’s own animal with its own properties whereas classic ‘blind’ links are nothing more than a road-sign with an address that may or may not still exist.
The way I’m using Z-codes is simply to make blind-linking more robust. I guess I never really thought about the codes representing an instance of a relationship between two things, which I suppose is closer to how a true Zettelkasten system works. That WOULD result in many additional codes assigned to the more popular targets as you describe. In that case I would start to just embed them within the files themselves.
I honestly haven’t ran up against that because my reference links aren’t nearly as intricate/circular as they’d be if I were trying to write a thesis or something. In that case I’d be trying to contain all the link relationships within software that handles two-way linking automatically.
Posted by Simon
Sep 13, 2018 at 07:08 AM
Similar to many of you, I’ve used many many many applications and workflows. What has stuck for me is simplicity. If the system isn’t simple, you won’t use it. Similar to nathanb I use a date stamp at the front of all documents. So for example a leaders agenda becomes 20180913-leaders agenda.txt. I also use tags in the name so a leaders agenda for my organisation (Hope Church) would be, 20180913-hc-leaders agenda.txt. I tend to write titles from the perspective of what will I look for in the future.
On the Mac Devonthink is my main repository for everything and has been for a decade. It has gigabytes of data strewn across 12 databases and I can pretty much find anything. Plus, all data can be exported so no lock in. I have 70K emails in Devonthink and it doesn’t blink and finds 10 year old emails with great ease. Plus the ability to configure the way the data is displayed is immense. I have smart folders set so that in my emails archive database there is a smart folder for each year from 2007 to 2018. This is just the beginning.
I also have a Finder repository that was started back in 2007 with AmberV’s comment from Literature & Latte linked in an earlier post in this thread. This is a simple date based repository. The folders all follow the pattern 2017 > qtr1; 2017 > qtr2, and so forth. I have a 30 day folder in which all my current work is saved. If a particular document is not modified in 30 days a background script adds the date/time stamp to the beginning of the document and files it in the relevant year and quarter. As I use area tags and descriptive titles I tend to be able to find most things in the system within 30 seconds going 11 years back when I started. It took some steeling of the nerves when I first started because up-un-till then everything was stored in subject based folders and dismantling that produced a large amount of fear. However, I’m totally happy with that system. Hats off to AmberV! I hardly ever file anything and as the script does everything and I only work from my 30 day folder period.
The quest to have everything in one place has led me to using many different programmes and methods and has in fact had the opposite effect of having many things in many places. I have come to the conclusion that my main problem is that I don’t stick with something long enough. The longer I stick with a workflow or system the more useful it becomes. The constant changing of systems does not allow my workflow to become mature. My greatest success in this area has come with using Workflowy for all my text based information. If I need something I wrote down it will be in Workflowy and finding hasn’t yet proved to be a problem, plus the tagging system is extremely helpful. I’m also beginning to experiment with Ulysses. I’m at a stage where I can see a clear distinction between my note taking, meeting notes, snippets of information and my research, teaching and preaching needs. I’m using Ulysses exclusively for preaching, teaching and research. This is working well as one of Ulysses’ strong points in my opinion is that it emulates the Silver searcher (Ag) found in a plugin in emacs. This means that Ulysses; when I search; not only displays the hierarchy of where the found item is located, but it also shows the line displaying the text snippet of my search criteria. Should the same search criteria be in more than on place in a single sheet of Ulysses, each line is shown. This really allows me to find a needle in a haystack.
My advice would be to find something that works and discipline yourself to stick with it for at least 6 months. Then you can begin tweaking.
Posted by Franz Grieser
Sep 13, 2018 at 12:20 PM
nathanb wrote:
>Sorry I wasn’t clear. I don’t have any filenames with more than one
>z-code. Maybe it’d be more accurate to say that the z-code is a special
>tracker for that file instead of the code representing a particular link
>between two things. In the case of only one note referring to only one
>file, the distinction is unnecessary.
>
>So if I have several notes/projects/tasks etc referring to the same
>file, it’s going to be with the same Z-code they all share.
I see. Thanks.
You also mentioned TagSpaces in the thread. Do you (still) use it? I find the concept appealing but wouldn’t want TagSpaces to mess with my files’ names. I know that the Pro edition uses a different mechanism but the free trial doesn’t.