Hi
first post from me but I have been lurking for a long time here. In real life I am an academic and as such am involved in many projects, some are in the acquisition phase (gathering information about funding agencies, collecting calls for funding programs etc), some are in the experimental phase (collecting and analyzing data), some are in paper writing stage, some involve only myself, some a few other people from my lab as well as other collaborators elsewhere, some involve expeditions, others conferences, students I supervise, theses to read, reviews to write, note-taking, idea-capturing, duration of projects ranges from weeks to years or decades.....you get the picture: THE perfect habitat to get hopelessly lost in. So far I managed to keep my head above the water but there must be better ways especially when it comes to choosing the tools to stay organized.

I know lots of you have experience with some of the programs I am using or evaluating in order to help me tame the chaos.
I have already tried the two extremes and found out that neither of these approaches work for me:
1) know one program well and use it for everything, even despite some weaknesses here and there
2) pick THE best software for each job and use each of these programs (and possibly more)
3) pick a really geeky program and get lost without getting anything done (actually, I achieved that goal pretty well, but now I know that I cannot master org-mode)

What are your suggestions for a good balance between points 1 and 2? I'd love to hear about lessons you learned from wrestling with similar problems, what are the fundamentally different tasks that you accept using different tools for?

Looking forward to hearing from you
Prion

Prion wrote:

I have already tried the two extremes and found out that neither of
these approaches work for me:
1) know one program well and use it for everything, even despite some
weaknesses here and there
2) pick THE best software for each job and use each of these programs
(and possibly more)
3) pick a really geeky program and get lost without getting anything
done (actually, I achieved that goal pretty well, but now I know that I
cannot master org-mode)

What are your suggestions for a good balance between points 1 and 2?
I'd love to hear about lessons you learned from wrestling with similar
problems, what are the fundamentally different tasks that you accept
using different tools for?

Hi Prion - welcome to the forum. As you say, it's horses for courses. But, here are my 2 cents, as a fellow academic type...

I think the balance lies in abandoning the focus on the tools and concentrating first on the workflows that need to be taken care of. Although the tools of course can't be separated from the workflows. However, a given workflow can be supported by different tools and toolchains (several tools strung together to take care of a workflow), and once you have worked out the workflows (process flows), individual pieces of the toolchains can be replaced.

So the questions are:

1) What workflows need to be set up to carry out your tasks?
2) How to model and construct process flows?
3) What tools are required for each workflow?
4) How to select the right tools for each workflow?

I'd suggest that it's also necessary to model/construct an overall hardware-software infrastructure (desktops, monitors, handhelds, input devices, software utilities etc.) to support the workflows. And I think there needs to be one overall general framework to take care of the overall backdrop and generic tasks.

E.g. by general framework I mean: What is the way new data gets into the system (email, browser, Dropbox etc.)? How are tasks created and tracked (calendar, todo list manager)? How are data captured, organised, labelled, retrieved (file managers, desktop search engines, databases)? How are projects identified, created and managed?

And then specialist (such as academic research, writing etc.) workflows and corresponding toolchains would need to be designed and strung together for each task.

Ideally of course one would want to have as few tools as possible, to make the process efficient. But it's unlikely that one tool can take care of the general infrastructure and project and data management and also of all specialist tasks.

How to model workflows and design toolchains? It can be drawn by hand or by using some kind of a concept mapping software like VUE. I like to save my VUE concept maps as PNG files and insert them into my database of choice, ConnectedText, so I don't forget them.

How to find the right tools? Through a lengthy process of search + trial. One needs to enter the CRIMPer life cycle. It can even take years of trying out different types of software until the optimal system and workflows and toolchains emerge and fall into place. Then the trialling and shopping can stop, and one enters into a stage of stabilising the system, tinkering on the edges, occasionally replacing tools or steps in the workflow (until the next crisis or technological paradigm change).

Dr Andus wrote:

How to model workflows and design toolchains? It can be drawn by hand or
by using some kind of a concept mapping software like VUE. I like to
save my VUE concept maps as PNG files and insert them into my database
of choice, ConnectedText, so I don't forget them.

Actually this for me is critical. Dealing with complexity and chaos means that the workflows and toolchains to deal with them also tend to be complex. I found out the hard way that if I don't record my process flow for a task that I have cracked, I might totally forget it (e.g. when I go on holiday) and have to keep wasting time with reinventing the wheel...

So I'm talking about the conscious maintenance of a meta level awareness of one's systems and having those systems visually modelled and recorded. These workflows and toolchains of course will keep changing but their evolution can be facilitated by managing this process consciously.

Some things to consider:
1. Few is better than many. (Do you really have the time to learn how to use a bunch?)
2. If you may need to use some material in the future, then the tool you save it in must either
a) Be able to export to some universal standard format that will always be updated for new
OSs.
or
b) Be a tool that has a long expected future--one that will be updated for new OSs.
3. Before committing to a tool, check our archives to make sure that it doesn't have problems
or seems to be on the verge of dying or is essentially dead. (E.g., Notemap on both counts.)
4. If you've read about what the people in this forum use, you know that virtually everyone uses
a different combination of tools. (Some, I think, use far too many.)
5. If you are going to use several tools, then Dr. Andus' workflow suggestion is important. You
don't want to spend half a lifetime searching for an item that you can't remember where you
put it.
6. Use consistent nomenclature, whether in a tree or in tags. Also, it is sometimes useful to
append to an item a few words describing its important content. This may make searching
a lot more productive.

Prion, first of all welcome, it's great to see long time 'lurkers' come forth. Apparently, this forum is bigger on the outside (to paraphrase Dr Who).

I don't have much to add to Dr Andus and Cassius' very good advice, other than to take a look at the recent thread on people's top 3 tools. I believe you'll find some great suggestions there and, most importantly, in context rather than in isolation.

Thanks for the welcome and your replies. I think I should clarify my intention and background a little.
Yes, few tools are better than many, at least if that's too many. With this thread I would like to encourage fellow CRIMPers with a roughly similar background to share their choice of tools and experiences. I am not new to dabbling around with tools (I should add that I am using a Mac) and trying to design workflows but each tool has a couple of strengths and weaknesses hence tools and workflows are intertwined and cannot be viewed in isolation.
One of the experiences that I have made is that the interface of a program is important to me. Sente for all its strengths (a wonderfully competent reference manager, the note taking and cloud syncing are working really well)....I found it intimidating in the long run and switched to Papers, which is less powerful but feels more balanced to me. The core functionality is there in Papers, there is little I do not use and little I am missing (with cloud syncing being the one big exception).

The tools I am using the most are in the heading of this post but I'd like to hear what you are using and how. I, too, am taking notes about my workflows. Whenever programs are concerned, I turn to a big Tinderbox document where I have stored this kind of information for a long time. Workflows about concrete projects are often stored in more or less free-form Curio documents although they can get too free-form at times for my liking. I am recently experimenting using Voodoopad for this purpose, too. It forces me to into a more linear structure and is a lot less spontaneous but that may actually be a good thing in the long run.

Tinderbox is another candidate on my list of things I am re-evaluating. I have a love and hate relationship with this program. It is uniquely powerful and when it works it is a joy but it also seduces me to waste days fiddling around with it. Its interface is often criticized for being not very Mac-like but I don't know. I find it not very inspiring but on the other hand some UI decisions are just wonderful (linking!). On the whole, though, I am nowhere near what Steve Zeoli and others can do with this program while experiencing a lot less friction, too.

I guess, the one thing I haven't quite decided on yet is where ultimately every piece of information is to be found in my system, the hub of my note-taking universe if you will. Right now, there are several locations I need to search, not-so-trusted systems living side by side on my computer. One way to simplify for me was to abandon taking notes about research papers inside my reference management program itself.

This post is getting too long already, I'll stop here

Best regards
Prion

Prion wrote:

I guess, the one thing I haven't quite decided on yet is where
ultimately every piece of information is to be found in my system, the
hub of my note-taking universe if you will. Right now, there are several
locations I need to search, not-so-trusted systems living side by side
on my computer. One way to simplify for me was to abandon taking notes
about research papers inside my reference management program itself.

I don't use a Mac, so I can't help on that front. But in terms of having everything (important) in one place, I think that can be achieved now, though it takes 1) the right software, 2) the right workflow, and then 3) discipline to stick with 1 & 2. In this regard I would suggest that a wiki can serve as such a system. It has two benefits for academics: 1) it can serve as your slip box (index card, Zettelkasten) for notes, quotes (basically everything essential), and 2) it can provide links to pretty much everything else that can't be imported for whatever reason.

Also, sometimes there are good reasons not to put everything into the same database. I keep all my academic etc. notes in ConnectedText, but I keep my academic journal PDFs in EndNote. I only link to those PDFs from CT that are truly relevant. So there is a virtual filter set up (in my mind), to separate the wheat from the chaff. For the same reason, I keep webpages in Surfulater, rather than in CT. I can link directly from CT to a Surfulater item, if necessary.

My main principle though is to bring everything important into CT in text form. For this reason I don't keep notes in my bibliographic manager (EndNote). Those notes would be important and there is not much one can do with them in EndNote, while in CT they can be linked, categorised, and found in searches. Similarly, although I read journal articles in PDF form, eventually I import selected quotes and annotations into CT. The point is to avoid duplication of effort. I found out the hard way that poor note-taking (keeping them in the PDFs or handwritten in books) results in having to do the work again...

I just remembered that I came across a Mac-based academic workflow here before:
http://drosophiliac.com/2012/09/an-academic-notetaking-workflow.html

For comparison, here is mine (very similar, just different tools):
http://drandus.wordpress.com/2013/01/10/academic-writing-workflow-with-connectedtext-freeplane-and-outline-4d/

Prion wrote:

I guess, the one thing I haven't quite decided on yet is where
ultimately every piece of information is to be found in my system, the
hub of my note-taking universe if you will. Right now, there are several
locations I need to search, not-so-trusted systems living side by side
on my computer. One way to simplify for me was to abandon taking notes
about research papers inside my reference management program itself.

Prion,

I am not an academic, and do not do much -- if anything -- that would qualify as scholarship. However, I am working on a local history book that requires me to capture notes and documents (mostly PDFs) for my research. I've tried to keep the workflow for this fairly simple: dump all research into Devonthink. Then pull what I need, as I need it, into Scrivener as I'm writing.

I don't have thousands of documents, so I don't even worry much about organizing the research in Devonthink other than I may add a tag or two, and I have a separate DT database for this project.

I will take a side trip into Tinderbox if I need to think through the logic or the flow of a chapter, and I've created a Map view of the timeline I'm dealing with, annotated with the primary sources I want to use for each period of time. And I may use Evernote as a ferry between my devices, if I happen to find something worthwhile when I am not actually at my MacBook -- which is frequently the case. If it is a document I find, I will put it in a special folder in Dropbox as the ferry to my MacBook/Devonthink.

While I have my issues with Devonthink (e.g., I hate the interface), it is the data manager I most trust on the Mac platform. I would use Yojimbo for this (because I like the cleaner interface), except that you can only have a single database, and I like keeping my projects separate.

Steve Z.

Thanks for the concrete suggestions, I actually saw some of them already but noticed some new ideas upon re-reading them, thanks.

I learned that there are some things I absolutely need to keep separate.

1) Task management is not the same thing as keeping a project structure. In the list of things that need to get done I just keep the tasks with very little background information. I know what project a certain task comes from when I see the task and new tasks are generated when reviewing the current status of each project with all their data and this takes place outside my task manager.
This was quite a liberating insight for me.

2) where to store pieces of information is not my biggest problem. Like Steve Z. I use Devonthink as my data warehouse. Other software would probably work, too, as long as it provided custom URL for every piece of information I deposit there, that is crucial for me and I use these custom links a lot.
Devonthink has a lot of things going for it but their implementation of tags drives me nuts. As a scientist set theory is probably deeply engrained in my brain and DTPO has a couple of oddities that makes using their tags like walking around with grit in my shoes.

3) The kind of information that I have the biggest problem finding a good home for is not the pieces of information as such but which connections I draw between them, this is separate from gathering the information in the first place and the most personal thing in the whole process. After a lot of trial and error I am coming to the conclusion that a wiki might be the best solution for this. Tinderbox, as nice as it may be otherwise, is too content with its splendid isolation to fulfill that role (I may keep using it for other jobs, though).
I evaluated Devonthink also as a candidate for the wiki, too, but although it knows some Wiki functionality (Wikilinks) it does not display backlinks, i.e. incoming links from other pages, which is crucial for me.
Voodoopad may not be as powerful as Tinderbox in that regard (what is?) but provides backlinks as well as custom URLs, talks to Spotlight, takes graphics more gracefully and may also scale better with size than Tinderbox.

Thanks for your comments so far.
Prion

Prion wrote:

I evaluated Devonthink also as a candidate for the wiki, too, but
although it knows some Wiki functionality (Wikilinks) it does not
display backlinks, i.e. incoming links from other pages, which is
crucial for me.

Check out Manfred Kuehn's blog. He's been using ConnectedText on his Mac and he has managed to integrate it with a number of other apps, incl. Devonthink (I think):

http://takingnotenow.blogspot.co.uk/2012/11/devonnote-and-connectedtext.html

Yes, I was aware of that and tested it, too. However, running two operating systems and relying on connectivity between programs on either OS is not my idea of long-term stability.

Prion wrote:

3) The kind of information that I have the biggest problem finding a
good home for is not the pieces of information as such but which
connections I draw between them, this is separate from gathering the
information in the first place and the most personal thing in the whole
process.

I also don't work on the Mac, and have not tested for myself the famous interoperability of applications. I just wonder, if you have more or less settled on Devonthink as your main repository, is it possible to use another tool to create relationships among Devonthink items? E.g., does Devonthink provide special links such as devothink:// which can directly open specific items?

My own choice for relating info would be TheBrain, which is available for Linux, Mac and Windows (runs on Java). The logic is very different to a wiki, as relationships are among whole items/entities, rather than via wikiwords integrated within the item texts. For me this provides a much better overview of connections.

Another strong contender (but less mature) is http://piggydb.net/ discussed here in the past. It provides wiki functionality, but also offers a way to relate whole entities among them via tags into visualisable relationships. I know my description is far from crystal clear, so you can see an example here: http://piggydb.jp/en/

Piggydb also runs on Java and is cross-platform. Additionally, it can run on a server which supports Java, and be accessible via LAN or the web, as is the above example. The Piggydb developer has a strong long term vision and is open to relevant suggestions.

3) The kind of information that I have the biggest problem finding a
good home for is not the pieces of information as such but which
connections I draw between them, this is separate from gathering the
information in the first place and the most personal thing in the whole
process. After a lot of trial and error I am coming to the conclusion
that a wiki might be the best solution for this. Tinderbox, as nice as
it may be otherwise, is too content with its splendid isolation to
fulfill that role (I may keep using it for other jobs, though).
I evaluated Devonthink also as a candidate for the wiki, too, but
although it knows some Wiki functionality (Wikilinks) it does not
display backlinks, i.e. incoming links from other pages, which is
crucial for me.
Voodoopad may not be as powerful as Tinderbox in that regard (what is?)
but provides backlinks as well as custom URLs, talks to Spotlight, takes
graphics more gracefully and may also scale better with size than
Tinderbox.

Thanks for your comments so far.
Prion

My takes:

- Wikis are bad: they will be broken if u move sth. So, never use them
- Future proof plain text (markdown) files are the best
- Keep all the text notes in the same folder, inside Dropbox
- access them by different apps (as the need arises).
- Use Simplenote as a glue among the apps
--- here is the step by step---
1. Create the notes in nvALT
2. Store them in a folder in Dropbox
3. Index the folder in Devonthink
4. Sync nvALT to Simplenote
5. Sync Tinderbox to Simplenote
6. Sync Scrivener to Simplenote

Work on any of the apps, as the need arises. The changes will be reflected everywhere (windows pcs, ipad, iphone, all fetch from the same folder).
- nvALT is the best for simple notes (drop what is in your head)
- Tinderbox is best for creating connections of ideas
- Scrivener is the best for drafting
It is possible to get the use the best of the apps without losing track. I hope this helps sb.

Dellu