Progam with QDA Qualitative Data Analysis features? Coding/tagging blocks of text?

Started by Carrot on 8/2/2012
Carrot 8/2/2012 3:42 am
I am analysing a large body of field-notes and interview notes in both English and Chinese for my PhD dissertation. I was thinking about using a QDA application such at AtlasTI, or HyperResearch to code blocks/selections of text and use this software to help me identify trends in the data, and to be able to call up all sections of similarly-coded data.

BUT the problem is this. The student versions of such QDA applications cost $100-200 and the normal-user versions are around  $600. And the programs basically all have time limits so it expires when student status is finished.  I for one, do not want to spend months or years inputting my data into one of these programs for my PhD and then have to spend $600 a few years down the road when I want to revisit my data.

Many many others feel the same way. ie) the guy who wrote WEFT, an open-source QDA
"Weft QDA was originally written out of curiosity whilst completing a Masters in Social Research Methods at the University of Surrey in 2004. I was annoyed by over-priced and over-complex commercial CAQDAS. The software was tidied up and documented, and then first released in 2005."
(Unfortunately, the developer of open-source WEFT QDA has been unable to continue work and has not found any group to pick up his project either. So, if you know anyone who wants to take up an amazing project, maybe we can help them restart.)

I am wondering if there is any application out there that has simple QDA (qualitative data analysis) features: namely the ability to:

select blocks of text and assign a code/tag to the block.
allow me to go through the document, and assign tags/codes to each section of text. T
program will allow me to search for all of the similarly-tagged blocks of text and see them in context

I have checked and found that the following applications do not have the ability to tag/coded of paragraphs or sections of text.: MyInfo, Ultra-Recall, InfoQube, KeepNote,

If you have any suggestions, for applications that can do this, or for other alternatives, I'd greatly appreciate your suggestions!
Alexander Deliyannis 8/2/2012 2:17 pm
If I understand correctly what you want to achieve, it seems similar to what was discussed in this thread http://www.outlinersoftware.com/topics/viewt/2666

In such a case, the recent Sense editor with Viewpoints support might be an option http://www.silvaelm.co.uk/


CRC 8/2/2012 4:41 pm
Well, there is always the CDC tool: http://www.cdc.gov/hiv/topics/surveillance/resources/software/answr/index.htm

Doesn't look like it has been updated since 2007.

Charles
Daly de Gagne 8/2/2012 5:30 pm
I wonder if a program such as Whiz Folders might suffice. I believe it allows linking to specific paragraphs in a document. It also has a system of tags. The program also allows for multiple open windows for editing and adding info, so that might also be helpful.

Daly

Carrot wrote:
I am analysing a large body of field-notes and interview notes in both English and
Chinese for my PhD dissertation. I was thinking about using a QDA application such at
AtlasTI, or HyperResearch to code blocks/selections of text and use this software to
help me identify trends in the data, and to be able to call up all sections of
similarly-coded data.

BUT the problem is this. The student versions of such QDA
applications cost $100-200 and the normal-user versions are around  $600. And
the programs basically all have time limits so it expires when student status is
finished.  I for one, do not want to spend months or years inputting my data into
one of these programs for my PhD and then have to spend $600 a few years down the road when
I want to revisit my data.

Many many others feel the same way. ie) the guy who wrote
WEFT, an open-source QDA
"Weft QDA was originally written out of curiosity whilst
completing a Masters in Social Research Methods at the University of Surrey in 2004. I
was annoyed by over-priced and over-complex commercial CAQDAS. The software was
tidied up and documented, and then first released in 2005."
(Unfortunately, the
developer of open-source WEFT QDA has been unable to continue work and has not found
any group to pick up his project either. So, if you know anyone who wants to take up an
amazing project, maybe we can help them restart.)

I am wondering if there is any
application out there that has simple QDA (qualitative data analysis) features:
namely the ability to:

select blocks of text and assign a code/tag to the block.

allow me to go through the document, and assign tags/codes to each section of text.
T
program will allow me to search for all of the similarly-tagged blocks of text and
see them in context

I have checked and found that the following applications do not
have the ability to tag/coded of paragraphs or sections of text.: MyInfo,
Ultra-Recall, InfoQube, KeepNote,

If you have any suggestions, for applications
that can do this, or for other alternatives, I'd greatly appreciate your
suggestions!
CRC 8/2/2012 6:41 pm
Carrot:

I'm assuming you are looking for some of the statistical functions built into a standard QA tool. The depth of that requirement may impact your ability to use the information management software that "Outliner Software" focuses on or whether you need the more robust capabilities of a true QA tool.

I'd hate to see you miss your degree because of a software malfunction :-)

Charles
Dr Andus 8/2/2012 8:20 pm
Carrot wrote:
I am wondering if there is any
application out there that has simple QDA (qualitative data analysis) features:
namely the ability to:

select blocks of text and assign a code/tag to the block.

allow me to go through the document, and assign tags/codes to each section of text.
T
program will allow me to search for all of the similarly-tagged blocks of text and
see them in context

I have checked and found that the following applications do not
have the ability to tag/coded of paragraphs or sections of text.:

This is exactly what I use ConnectedText for. I can't link right now (writing on my Palm) but if you search for CT here you will find my case study.
Cassius 8/3/2012 12:46 am
I have no idea if this will help, but it is worth checking out. The R project is devoted to the development of a wide range of FREE statistical and analysis tools. Originally based on the S+ system, it continues to grow with new applications being continuously added.

See http://www.r-project.org/
Carrot 8/3/2012 3:07 pm
Hi Charles,

Thanks for the reply.
I think that for now, I do need the more robust capabilities of a QDA application.
Luckily I still have my version of AtlasTI 6 that runs well. I'll use it for now and will explore Dr. Andus' suggestions for using ConnectedText http://www.outlinersoftware.com/topics/viewt/3799/40

- C

CRC wrote:
I'm assuming you are looking for some of the statistical functions built
into a standard QA tool. The depth of that requirement may impact your ability to use
the information management software that "Outliner Software" focuses on or whether
you need the more robust capabilities of a true QA tool.

I'd hate to see you miss your
degree because of a software malfunction :-)

Charles
Carrot 8/3/2012 3:14 pm


Dr Andus wrote:

This is exactly what I use
ConnectedText for. I can't link right now (writing on my Palm) but if you search for CT
here you will find my case study.

Dr. Andus,

Thanks very much for your reply. I looked at your posting about ConnectedText http://www.outlinersoftware.com/topics/viewt/3799/40
I'm wondering if you might possibly make a short video outlining the process, or perhaps illustrate the instructions you provided with screenshots?
I'm sure many of us would greatly benefit from that.

What you did with CT sounds exactly what I should be doing too. Now the trick is to see if I can understand how to replicate it.
One problems- I really do not like the idea of using markup language - It just doesn't seem to make sense to me to use that (I simply have too many things to try to learn and remember- why not just click on toolbar icons?)

I'm very curious- do you think I should begin using ConnectedText rather than AtlasTI? I have about 18 documents each about 100 pages, each one my fieldnotes for different research trips.
Thank you!

Carrot


Alexander Deliyannis 8/3/2012 4:52 pm
Carrot wrote:
I'm wondering if you might possibly make a short video outlining the process, or perhaps
illustrate the instructions you provided with screenshots?
I'm sure many of us would greatly benefit from that.

YES YES YES
Dr Andus 8/3/2012 9:02 pm
Carrot wrote:
I'm
wondering if you might possibly make a short video outlining the process, or perhaps
illustrate the instructions you provided with screenshots?
One
problems- I really do not like the idea of using markup language - It just doesn't seem
to make sense to me to use that (I simply have too many things to try to learn and
remember- why not just click on toolbar icons?)
I'm very curious- do you think I
should begin using ConnectedText rather than AtlasTI? I have about 18 documents each
about 100 pages, each one my fieldnotes for different research trips.

I'd love to but unfortunately I'm travelling until end of Aug. w/o proper PC or internet connection.

Re CT, there is a bit of a learning curve but so is w/ Atlas or NVivo. As for markup, you literally only need 3 or 4 codes to do what I described. I believe it's faster to type == than to select text and click on an icon to code something.

Whether Atlas or CT that's depends on your personal preferences and whether you're willing to put in a couple of days to learn CT. I couldn't quite warm to Atlas and preferred NVivo. However CT is just far more flexible and I can import masses of data and organise it the way my mind works. It is essential to sign up to the CT forum and read around and ask for help.
Fredy 8/3/2012 9:37 pm
Carrot, you say you will probably use such a program for years. Let's suppose after your PhD you'll be well paid. You think 1-2 Benjamins now and 6 others then will be too much money, for such "overpriced" sw, and you say many, many others think as you do. Some weeks ago, in the UltraRecall forum I (as schferk) explained in depth WHY sw developers have lost all interest in refining their babies. People who think like you are the culprits, I fear, for such a development to extreme poverty in functionality of available sw, and, as I explained there also, the cloud crowd will not be better off neither, all to the contrary.

Daly, did it ever occur to you what the original idea behind a citation function in a forum sw was? Just a hint: SELECT BITS to comment on those bits.

ConnectedText, it's a pleasure to see you're considered here, lately, as the Swiss pocket knife to any text processing problem whatsoever; it's heart-warming to see one developer flourish at least, even if it's for the wrong reasons. At soon as you'll be able to prepare my coffee and do my laundry, let me know, I'll happily buy.

Carrot, if you really want to become independent of your proprietary "format", may it for bucks or for other factors: Consider using a programmable editor. 1 line 1 paragraph. Each line starting with at least one code "super-paragraph" (this can be automatted), more codes at libitum, here or at the end of lines or anywhere within, just use dollar, pound, whatever leading signs.

Many such editors can sort lines by block content, i.e. chars x to y of these lines (hence the utility to carefully design your codes('lengths, incl. blank codes)), not just by their start chars. Many such editors can filter lines by multiple criteria (= codes = "fields" with specific content); regex goes without saying. They're all many years old - so what, except for Unicode, perhaps?

There are some special editors like KEdit that seem to facilitate such filtering at first sight - I'm not advocating these traps, though.

You can buy for cheap such editors, or even "steal" them. All that's needed for everything you'll ever need here is perhaps one day of learning the corresponding syntax, incl. some statistical functionality that might be needed.

Then, with your PhD, you'll probably work within a corporation that buys some sw, fortunately. Corporations buy sw, individuals by iPads. (800$ every years's nothing for such lifestyle hardware, but beware of applics costing 5$ - they must be 1,50 or they will be unacceptable). That's why in many years here, nothing really new here, and nothing will come.
Carrot 8/4/2012 2:28 am


Dr Andus wrote:

Re CT, there is a bit of a learning curve but so is w/ Atlas or
NVivo. As for markup, you literally only need 3 or 4 codes to do what I described. I
believe it's faster to type == than to select text and click on an icon to code
something.

Whether Atlas or CT that's depends on your personal preferences and
whether you're willing to put in a couple of days to learn CT. I couldn't quite warm to
Atlas and preferred NVivo. However CT is just far more flexible and I can import masses
of data and organise it the way my mind works. It is essential to sign up to the CT forum
and read around and ask for help.

Thank you for your reply!
I'll have to experiment more before I can decide which application might be better for my work.
I've found ConnectedText quite difficult to figure out.
Your approach sounds quite intriguing and one day, if you have time to make a video or illustrated
set of instructions, I'm sure many users would find great use in it.

You'd mentioned you code your document by adding Headings (up to 5 levels)
I think this will be the major problem- I need to use about 80 different codes to follow different themes in my work.
Perhaps CT will not be suitable for this job, but I'll purchase a copy and learn how to use it for my writing and other projects.
The other similar open-source solutions (OutWiker, Wikipad, Book-On-a-Stick) are progressing wonderfully, but do not yet seem capable of providing the functions I will probably need.

For now, I will continue to use AtlasTI and experiment with HyperResearch QDA.

-C.
Alexander Deliyannis 8/4/2012 8:49 am
Fredy wrote:
Carrot, you say you will probably use such a program for years. Let's suppose after
your PhD you'll be well paid. You think 1-2 Benjamins now and 6 others then will be too
much money, for such "overpriced" sw, and you say many, many others think as you do.
Some weeks ago, in the UltraRecall forum I (as schferk) explained in depth WHY sw
developers have lost all interest in refining their babies. People who think like you
are the culprits, I fear, for such a development to extreme poverty in functionality
of available sw, and, as I explained there also, the cloud crowd will not be better off
neither, all to the contrary.

Though I agree in principle that we should be willing to pay for the tools of our trade, and as a chronic CRIMPer I've paid quite a bit more, I have to note that the pricing of software overall seems to me extremely irrational and in many cases unjustifiable. Moreover, I have seen no positive correlation between the price of software, the support provided and its long term viability. In fact, I have seen several instances that could support the negative correlation of the above, i.e. more affordable software is often better supported and available in the longer run.

Examples:

- Notemap, by far the most expensive single pane outliner, whose development has stopped ages ago, and even basic bugs have not been resolved (as Cassius has pointed out on several occasions)

+ MyInfo, a very moderately priced two-pane outliner, continuously updated and excellently supported. There are actually several other positive examples in this category, and I believe that it is the development and support which keep users faithful to any one such program, rather than the (relatively few) differences in features.

- MindManager, the first and arguably most successful mind mapping program, significantly overpriced for what it does (especially given the many alternatives now in existence), now in its 10th incarnation, has maintained its outline mode crippled as if it were an afterthought, and has even lost useful features such as OPML support along the way. You may see here http://www.mindjet.com/pdf/us/MM2012_WinFamilyComparison.pdf and judge whether the development from one version to the other would justify the upgrade price.

+ Noteliner, a modestly promoted but very powerful and completely free program, created and maintained as a labour of love, gaining strong features along the way, several of which have been proposed by it users in this here forum. Again, there are other such examples, like the very special TreeSheets.
Dr Andus 8/4/2012 11:57 am
Carrot wrote:
You'd
mentioned you code your document by adding Headings (up to 5 levels)
I think this will
be the major problem- I need to use about 80 different codes to follow different themes
in my work.
Perhaps CT will not be suitable for this job, but I'll purchase a copy and
learn how to use it for my writing and other projects.

There are several other features in CT that can be used for coding, the Headings are just one of them. I also use the Categories, Attributes, and Properties. The last two allow for some complex inline and search queries for building reports. It probably also depends on your particular qualitative research philosophy and methodology as to how you would want to use these for your purposes to arrive at a desired outcome.

For now, I
will continue to use AtlasTI and experiment with HyperResearch QDA.

I think it is a good idea to reallly learn and use one QDA extensively, and then once you've learnt the most useful features and got frustrated with the limitations, then trying to remodel your process in CT. I used NVivo for 6 months daily and processed half of my data before I got fed up. Ironically what got to me that I ended up with 100 codes and it became difficult to say which were the important to focus on.

Sorry I can't do the detailed instructions for you right now for CT. I'll try to do them when I get back home in Sept. Good luck!
Carrot 8/4/2012 2:48 pm


Dr Andus wrote:
Carrot wrote:
>You'd
>mentioned you code your document by adding Headings (up to 5
levels)
>I think this will
>be the major problem- I need to use about 80 different
codes to follow different themes
>in my work.
>Perhaps CT will not be suitable for
this job, but I'll purchase a copy and
>learn how to use it for my writing and other
projects.

There are several other features in CT that can be used for coding, the
Headings are just one of them. I also use the Categories, Attributes, and Properties.
The last two allow for some complex inline and search queries for building reports. It
probably also depends on your particular qualitative research philosophy and
methodology as to how you would want to use these for your purposes to arrive at a
desired outcome.

>For now, I
>will continue to use AtlasTI and experiment with
HyperResearch QDA.

I think it is a good idea to reallly learn and use one QDA
extensively, and then once you've learnt the most useful features and got frustrated
with the limitations, then trying to remodel your process in CT. I used NVivo for 6
months daily and processed half of my data before I got fed up. Ironically what got to me
that I ended up with 100 codes and it became difficult to say which were the important to
focus on.

Sorry I can't do the detailed instructions for you right now for CT. I'll
try to do them when I get back home in Sept. Good luck!

Thanks for your reply!
Now I am even more curious about your method of using CT.
I'd much rather learn how to use an easily affordable program like CT and use it for my huge projects.
As for my "qualitative research philosophy", I don't really have one- just sort-of a grounded-theory
approach to code the data based on concepts that come out of the interview material. Once the data is coded,
then to see what codes overlap. I'm sure the process will teach me a great deal about themes in my data.

AtlasTI is powerful and well-designed and since I've already paid for it, it would behove me to learn it well.
Once I've done most of my work, later I will go back and figure out how to do it again in CT.
That might be a nice way of creating a website for my field research, and a way to link together the photos, video and the story
I want to tell.

The problem is now I find CT quite difficult to figure out.
Please do make that video when you have time in September. I think a lot of us would appreciate it greatly. I a lot of the people who use the CT forum would like that too.


Dr Andus 8/4/2012 5:42 pm
Carrot wrote:
As for
my "qualitative research philosophy", I don't really have one- just sort-of a
grounded-theory
approach to code the data based on concepts that come out of the
interview material. Once the data is coded,
then to see what codes overlap. I'm sure
the process will teach me a great deal about themes in my data.

That's pretty much my approach (a grounds-up ethnographic approach).

I forgot to mention that the "Include" feature is another critical element of CT's analytical capabilities. It allows you to extract bits of a document and include them in another document. So you can collect similarly themed (coded) sub-headings from several documents and construct a new page out of them. This is what I call abstraction by way of extraction.

As for getting started with CT, it's been already discussed on another thread here. It's the most difficult aspect of CT because it's somewhat idiosyncratic, plus being a generalist wiki, it's probably impossible to include instructions for all potential uses. It took me several years and several false starts before I realised what CT is and how I can use it. Interestingly though, after the initial breakthrough it was very quick to get up and running. Folks at the CT forum have been extremely helpful. One just needs to get stuck in and start doing things and start working through problems and learn it step by step.

I'm obviously a big fan, although I've only been using it since March. But it's kind of saving my life because I was drowning under data and didn't know how to pull everything together for my PhD project. I was expecting NVivo to do that for me but it wasn't right.
Dr Andus 8/4/2012 5:54 pm
Fredy wrote:
ConnectedText, it's a pleasure to see you're considered here, lately,
as the Swiss pocket knife to any text processing problem whatsoever; it's
heart-warming to see one developer flourish at least, even if it's for the wrong
reasons. At soon as you'll be able to prepare my coffee and do my laundry, let me know,
I'll happily buy.

Fredy, I sense some irony in your words. Actually I only use CT for one thing: as the main tool for my PhD project, so I'm not claiming that it can solve every text processing problem in the world. Having said that, it seems to be a very versatile software. After all it is a wiki, so it depends on the user's imagination as to what one makes of it and with it (kind of like the internet).
Fredy 8/7/2012 3:17 pm
Doctorandus, glad you overcame your procrastination issue. ;-)

Alec, you're right, there is no correlation between sw price and help / service / development issues at first sight since there are notable exceptions, and other factors that blur the picture. Then, I didn't pretend there was, but I differentiated - not clearly enough I must say - between "corporate" sw (NoteMap, MindManager - sw bought by (often large) corporations or by legal entities in order to do business work - sometimes they ar not well served, as in the case of NoteMap, but they prefer buying third-rate sw from their usual source: here, LexisNexis, instead of buying much better sw from "less reliable sources") and sw bought by individuals (may they be able to deduct it from their tax bill or not).

And I said, or wanted to say, these have become two different markets, with rather tiny intersection, in fact, less and less so. And within that second (!) market, most developers have ceased to do serious work, and I pretend that's for complete disenchantment with individual buyers' buying habits. (MS Office Student & Home = about 90 dollars for the whole family: 3x Word, 3x Excel, 3x their outliner nobody wants... and without Outlook, e.g., let alone Access - compare with 1x (!) Word alone, ca. 1990, ca. 900 DM = ca. 1200 euro today's worth.)

In such an environment (of which the applics market is the most obscene deformity), individual developers ain't able anymore to insist on a bit viable prices, and only a few succeed with replacing this "individuals" market by making an access to the "corporation" market, many of them for lack of quality, and almost all of them getting withheld by the above-described phenomenon of corporations preferring lesser sw by more renowned developers (hence the invariable "We'll give it to the developers" in 1-man developers' help forums).

So there are some niches left: crm, macros / text expanders, Outlook add-ons, and several others - editors were such a niche decades ago.

Btw, your MyInfo example is a perfect one for SIMILI-development, in order to please people that are not too demanding = developers pretending only to listen to their customers (and it's a good example for another phenomenon I saw here: charging in dollars, being in East Europe, while corresponding U.S. "addresses" give up):

- hadn't second pane; is with one for some 18 months now, not editable and not even dockable so that either you buy a second monitor in order to use it or go bonkers
- hadn't good print / export facilities but pretended to have these within the help file only; the corresponding features couldn't be checked for since they were greyed out in the trial; after buying, Petko said, oh yes, was planned (= rare example of a help file being ahead of actual development when it's normally the other way round), will be done; Petko 6 months later: have no intention to do anything about these things; 2 years later: nothing been done
- had good special feature: possibility to display first 3 lines of hit items within the (search) hit table; was abolished without further notice; a "good" customer = one that even wrote a routine for importing askSam files into MI, finally asked what was going on; Petko: will be re-introduced later; feature is missing for 18 months now; say you run your little business on MI, say you rested on that fine feature (customer "Smith" calls; you search for that customer; you get 15 hits; the info within those first 3 lines will tell you which of those 15 hits to "open"); you try to revert to the (tremendously buggy) old version: I fear it's even another format so you can't revert without losing data?
- old version tremendously buggy, as said; many intermediate versions; all "beta" with timeout; new PAID version, so, after many versions to do away with innumerable bugs, perhaps with your big help (and certainly with your amounts of patience), you either pay, or you revert to your old, bug-ridden paid version: NO free intermediate "minor" updates then, contrary what you expected and were made to believe
- paid new version extremely bug-ridden; let's hope this time people get free intermediate minor updates; I didn't get any
- tags, yes; quick filter your items by just two tags, no
- attributes, yes; quick filter by more than one attribute at a time, no
- yes, there are some weird ways to do such a filtering, with LOTS of key pressings / mouse clicks, but it's so devoid of practicability that your trying to use MI in a "productive" environment is technically possible, but it's forcing you to do a lot of unnecessary work again and again, and it's forcing your customers to wait for your doing so, on the phone, or in front of you; I call this unacceptable
- no Boolean search of course
- asked for a little pm system; have a look at what has been done
- I could bring many more examples, I'm just reviewing some nuisances from memory
- and yes, they introduced hoisting and amended clones creation = more or less, two years' work - and for legal reasons, you need to call this "active development".

Which is to say, our MI example is a perfect one in order to show WHY corporations refrain from buying sw from pop-n-mom affairs: The TOC simply will be much too high. Let alone our choice exmple, askSam, many brilliant ideas from ancient times, and so bug-ridden that anybody considering the use in any "productive" environment can't sleep anymore for fear of total data corruption.

And yes, there are a few exceptions, as to most rules. Some individuals doing brilliant little sw for their own use, and for the benefit of others. Those are rare examples. Back to NoteMap: One-pane outliners ain't viable, neither in a legal environment, nor anywhere else. No regrets, then. But askSam was finally bought / overtaken by its former marketing chief... instead of somebody who'd been able to make it corporation-ready: We all would have greatly profited by such a move. Or more clearly: With sw, ask yourself, are they able to rely upon an existing, large "corporate install base". If not, let'em die without your commitment (data, time, problems, the money being the lightest part in all this).
Fredy 8/7/2012 4:18 pm
Editors are much undervalued. Most people don't realize that one of the beauties of editors is, you have data you can work upon in many different such tools, i.e. with editor 1 you do this, with editor 2 you do that - and you can automate / script almost everything, within those editors as in the concertation of such a workflow.

To whom it may concern, some hints:

- don't be afraid of subroutines having to check for things or process 10k of lines, one after the other; a good editor will only take some seconds for millions of such computations...

- incl. checking checkboxes or such, i.e. you do (complete or partial) computations on / in a five-digit number of "lines" = records, all writing intermediate results into up-front "fields" of these lines, e.g. IF in line x ... and ... and ... (= could be 50 or more checks, comparisons, whatever) THEN fill up the "field" $e at the beginning of that line with "1"; ELSE fill it up with (or leave it at) "0", so you would have a line starting with $e1;other field;other field, etc. or with $e0, etc., etc.

- the same works then for sorting / filtering, i.e. even if you have got "fields" within your "text" within these lines = records, you can SORT by these criteria, just check, and if yes, fill out a special "field" in front of your line; as said, the subroutine doing that special filling out (or a reset to zero there) would be perhaps 2 sec. for 10k of lines or even less on a modern pc

- you can do this with SEVERAL such "check fields" (or even with "de-doubled" data "fields": Whenever a data "field" within a line becomes relevant for sorting, just de-double it in front of the line), and you know you can sort by several criteria, a, b, c: sort by c, then by b, then by a, and you'll get a body sorted by a then b, then c; combine this with filtering and you'll get only the relevant records (to check visually, to export, to print...), and in triply sorted order - remember for tenfold sorting, you can quickly write a routine to do it, then just fill in the criteria each time

- such de-doubling of data is one means to avoid having to cope with leading zeros, for example; if you've got 1- to 3 digit numbers, you'll need subroutines putting zeros in front of 1- and 2-digit numbers, of course, if you want to sort by "fields" containing such numbers, or if you want to "block sort", i.e. to sort by field contents AFTER such number "fields" - you avoid this by de-doubling such "fields" containing "regular" numbers (= without leading zeros) with special "fields" where you put such leading zeros where needed, but leaving the originals fields intact, OR you could use de-doubled number fields without leading zeros, but with the original numbers PLUS, say, 1,000: if you have numbers having 1- to 3-digits (1 to 999), add 1000 within the special field, so you'll get invariably 4-digit numbers; the same goes for "original" data under the condition you'll never forget which range the "real" original data was in, i.e. subtraction needed before publication...

- I spoke of statistical analysis when most editors just have rather basic number crunching. No problem, just remember, do everything as simply as possible WITHIN the records, i.e. put intermediate results there, and then, on these, "run" Excel or special sw (but don't try to do text processing within Excel, and XL Notes' a dead end - willing to put 10k of text paragraphs into 10k of XL Notes' Word files within Excel cells? See what I mean when I so often declare things as "amateurish"?)

- It goes without saying that lots of text analysis can be half-automatted, i.e. if x near y and perhaps even z in th line then... and then, you'll have that filtering in order to visually check (and undo your unwanted field "yes" or whatever settings by such rules): It's far better to have 200 such "fields" set up automatically (or their content set to 1 from 0 or whatever), and have to check (in a filter table on the screen) and put off 10 such "fields" / settings manually then, than to code manually 190 of such records, out of perhaps 2000, in a pro like CT or whatever

- Even your Excel (or whatever your statistic tool is) results can then be re-introduced into your editor file, be it in front (append), or be it by replacing the values of the corresponding (let's say first dozen or so of your) "fields" there, here again, a combination of "block processing" and the Excel export format can do wonders.

- For 80 or 180 codes, an editor is brilliant, especially so that whenever you see that you cannot preserve your code $e but need codes $ea, $et, etc., etc., there is "global replace", when, as said, and that's the thing non-editor users first must become aware of, 99 % of all processing will be done WITHIN single lines, but for 10k of lines 10k of times in a row - that's what editors are for when doing data processing.

- Of course, for reading bits, you toggle to "word wrap" (buy a large screen anyway); the de-doubling of "fields", btw, is a good way to "read" things = check visually for things on the screen, even for "field" contents where these original "fields" are further away "down into the length of the line"; you could even have some 3 or 4 "read fields" rather in front, into which at any given time you copy the contents of various, interesting fields upon which you wanna check.

- In my first post on this I meant, have your original data preserved in original form, i.e. if you have let's say 50 conversation with 50 Mali women (or with whomever), do special lines ##mw01 or whatever, and then do a line for every paragraph in that conversation, in the form #mw01£001$code1$code1... up to #mw01£999$code1..... for (in this example) up to 1,000 lines for each conversation, etc., etc. i.e. have a "natural sort" of your original material you can anytime revert back to, and of course, and such editor can easily number those lines / paragraphs of each conversation or whatever then. So the very first such "fields" would be invariable, but as said, if you assure these start "fields" are of equal length, char-wise, you easily can sort on the followings fields

- Must I really add that you could use another special char whenever you must / want to combine several paragraphs in one line, and which will allow you for dissecting them automatically, afterwards? The same goes for text formatting (within the original, to be preserved, or for better presentation afterwards) - people doing stuff with CT don't need the slightest hint from me how to do this, within an editor just the same.

- Etc, etc., there are lots of more possible hints. As soon as you got the strength of editors, many of commercial sw's will become not only obsolete for you, but you will discard them for total inacceptability: An editor will do whatever you want it to do for you; commercial sw, most of the time, is "dumb crap" in comparison, or costs some Benjamins I'd be willing to spend... but then, I'd fear that quickly, I'd come to that wall beyond which the developers didn't to their work as I want their product to be - whilst with an editor, I add a line a two to the script, and I'm done.

- Re KEdit: For a start with filtering, this program is pure gold; it's just for more elaborate tasks I prefer better stuff.

- Re askSam: The beauty in that prog was that it was a "real application, ready for use", and it had lots of editor-like features coming with such a "general public program". Whilst en editor is an editor: beauty on the screen there is not.

But, Carrot, don't you think that for purely esthetical reasons here, you, like Daly and some others, should avoid endless "citations" when they ain't of any use?

Daly de Gagne 8/8/2012 1:40 pm
Fredy, I've noted - twice, now - your request I not quote large blocks of a previous post when to do so isn't relevant. You are quite right, about that, and I apologize to all concerned.

For my part, as one who appreciates what you have to say and your insights, I'd ask if you could take the time to edit what you write so you can say the same thing in perhaps half the space. That would be a consideration for all of us who wish to read what you write.

In terms of MyInfo, just one comment.

You refer to the second pane, that is the pane which allows one to see the contents of one note while working on another note. You are correct in stating that it is not editable. From the get-go Petko said in this version of MI it would not be editable, but that in the next major upgrade he would add that feature. I agree it would be nice to have a pinning feature.

I give credit to Petko for at least adding the ability to have multiple windows, even if not editable at this time. I suspect most developers would have just held off adding the feature until they could include the editing capability. For research and writing I find multiple windows extremely useful, and look forward to the next major upgrade when Petko will add editing features. This would put MI on a par with Whiz Folders in terms of this feature.

Daly


Fredy 8/12/2012 12:28 pm
General :

Daly, I know you're right... Problem is, I'm developing ideas here (and in other forums) on the screen, much better than I'd do without being online (which is a very interesting aspect since I'm against online file storage, etc. - "interesting" meaning I didn't get it yet; another aspect of this phenomenon: I sometimes vision films online - and prefer to do so - even when I've got the same film on DVD in better quality - that's completely crazy, I know. Similar for TV (I haven't got a TV set for 20 or 25 years anymore in order to avoid all that crap there (and in absence of a mind strong enough to simply switch off all that crap...) for many people: There's something in the mind of many people that urges them to PARTICIPATE IN SOMETHING RUNNING / going on at this very moment, or anything like that, and writing - "direct" - to a forum triggers ideas I wouldn't have when that same forum sits just waiting for un upload. I SAID it's crazy, but perhaps, if we fully understand what's going on, we could overcome tv addiction of myriads of couch potatoes visioning pure sh** on tv? ( So I tried, with my last thread, to do some cutting up into several posts beforehand, but saw again that some ideas had to be referred to in several context, this way, when, with an editing function, I could have done better: So Chris' the culprit! ;-) )


To Daly and MI users only :

On Petko's second frame. It's not the missing editability of it that bothers me so much (and hey, we've got 18 more months now!) but (I don't follow MI's development narrowly in detail, just checked in January again for that feature) the impossibility to have it non-vanishing and dockable (cf. hits table for search functions, where MI is a prominent example indeed HOW to do things right!): You can't even use it for some "reference table" or such things since it does not only overlay your normal MI window (and thus hiding that stuff you need to see!, instead of you being able to dock it somewhere WITHIN that window), but also, it vanishes whenever you go to another item (within your MI window), and that's just another example of Petko's way of giving in to his users' wishes: Yes, he does something, but it's so crude it's almost unusable and more or less devoid of any usefulness for anthing (and to do this specific thing right would have asked for 10 minutes of programming - and I know about these things).

I give a real-world example (and I said it in January, within the MI forum, re developers not even using their own progs, to all evidence!!!): You got a client calling. You search for his main item. Then you want to see his contact history. You pour his main item into the second pane, then navigate to his second (or whatever detail) item in your main screen... Second pane: vanished. You get it anew. Then you speak with your customer about some details; you need to refer to the third (or whatever) item of that customer (let's assume he's got one main item, then some sub-items in your system, so the navigational issues ain't of any difficulty): Again, your "reference" pane vanishes. And so on, and so on, because Petko's avoiding these additional 10 minutes' programming.

This in unaceeptable from a "philosophical" point of view, it's this "good enough for my customers as it is" stance that's inacceptable for me. Your argument of "at least this when another guy would have waited upon full functionality" is due some respect and goes for the missing editability (since here, more programming to do, but not worth 2 years or more - I had multiple editability in my outliner 15 years ago, as many, many other elaborate things, and the whole thing was about 1 man year, so I have means to assess how many hours this or that task will take (and take from an experienced programmer like Petko that is). And then, lots of people asked for that pane again and again, so Petko "bought piece" from introducing such a basic pane, but please trust me, from a programming pov, introducing SUCH a pane, with SUCH (lack of) functionality, I, as a non-programmer (and if I know the source code, i.e. be accustomed to it, or have the developer by my side in order for him to "navigate" me), can introduce THAT feature as it is (= as it was in January) in 30 minutes, and no, that's not another one of my "overdrawings to put things into perspective", I'm talking of real 30 minutes, and it'd be done!

So in the end, it's just another example of, give this little to customers, and they be quiet! (And I bought 5.07, and then all I / we got was time-limited betas, instead of getting rid of numerous bugs within the paid major version, and that's unacceptable, too - not for new functionality, that Petko could have taken out off a final version 5.9, but leaving all the bugs in... so that was another education for me: 5.07 = final version for the whole number, lesson: When buying sw, have a look at the former numerotation policy of the developer in question and don't extrapolate from normal numerotation policies.) (Petko's enhancement of the cloning function was very good (and aligned an almost unusable function to the standard set by UR - I see good ergonomics when there are any.)

Daly, you're one of those people who are kind to people - not everyone is at all times. But by letting pass too much sloppiness in programming, you invite / drive developers even more into not doing their dues, and, as I explained here these last days, they've got a quite natural propensiveness to let the clutch slip even when let alone - bad or very bad sw is the result. ;-)
Fredy 8/12/2012 12:59 pm
EDIT General :

The same goes for music: I've got so much music on cd (and lp, yeah, old men and their juvenile purchases) - and then, I listen that very same music on youtube, and in my weird mind, that's "nearly live" whilst my music, in much better quality, on physical supports, I consider "dead" - more joy with lesser quality "servings coming from somewhere" than with (sometimes) top-notch recordings here at mine's = dissected from the world outside. As said, if we can fully explain what's going on there, perhaps we could heal couch addictiveness for multi-million of people.

But with iPhones, iPads, etc., it seems to get "worse" even: Your own data - identical data -, when it comes flowing from the cloud back to your on-the-road devices, seems to get another quality for yourself than that same (your own) data could have if you just synched your stuff (by cable or by common wlan, etc.) within your own home, from your desktop pc there. Muse a second about it - we're at the core here of 90 p.c. of the cloud phenomenon - only about 10 p.c. of cloud data being data synched with others and thus representing the original idea behind the cloud -

90 p.c. of the cloud functions for the bad reasons - and it functions extraordinarily well with those 90 p.c. - but nobody, to my knowledge, has ever verbalized this irony, let alone explained it (which I don't pretend to do). Your own data, I say, getting a new quality when synched cloud-wise, and it seems people GET something new out of it only the cloud can offer, with material then that is bit-wise identical. If that's not crazy, as the Germans say...


Fredy 8/12/2012 1:35 pm
Sorry for multiplying bits: It's not the music videos, many songs come without any video there. And it's not cloud synching beforehand, before leaving home. It's the "instant availability on the road", i.e. synching on the spot, when you're in need (or just in "need") of the data in question. It must be something like "getting something from out there at this very moment" - to put it bluntly, even data identical to one that is on the harddisk of your device already. "Getting something from the television" must be something like "getting heroin from your dealer" - I'm sure it triggers some special functions in your brain that ain't triggered but the material you've already got there.

And now for forging the bridge to offerings like The Brain. Could it be that when musing on your own Mind Maps (Tony Buzan's rights reserved), your own items there trigger something that your own lists don't trigger, because of the blank space... or because of the (more or less aleatoric?) spatial representation (it doesn't work with alphabetically (= systematic-wise cut-up, let alone systematic) lists, so the common "space" notion (in mind maps, in cloud services) seems to be important - tv? by cable, though? But what's at the other end of the cable? CHAOS, from your pov. I wanna say, things you own have a dedicated place, even if you don't remember that specific place at a given moment. Thus, they're "dead" in a way. Things in the cloud - or within the tv set - don't have a place known to you / inscribed in your mental representation of "what's where".

So there see to be two factors at least, concurring (and with some others probably): The instant delivery (instead of "it's all there already"), and the spatial whereabouts (= you don't "know", but you "discover on delivery" or something). (And then, mind maps are multiple times more successful than The Brain - but in mind maps, if you don't touch a file, it's spatial representation will not be changed from last time, whilst in The Brain, it's all flurring arround all the time - so people seem to prefer SOME stability beyond that "don't know where it exactly is" curtain...)

And so on... Lot's of possible factors to muse on, beyond the "share" factor, which is not to be left behind, though. Watching TV, listening to youtube means, (at least in theory, for youtube,) others could listen / watch the same material as you do (but with youtube, probably at a differed time rate (you're at 5''23, others might be at 2''45) - tv only is synchronized time-wise. So there seems a be some "sharing" factor even if it's not about REAL sharing of your own data on the services you hope they will NOT share it against your will... Don't know, but there is the phenomenon that millions of people prefer the net even when they don't really need it, myself included when thinking. So there is some adequacy in trying to think about what's going on with all that "sharing" business especially so when it's NOT about real sharing.

(Unable to edit: It's all too new for me.)

Fredy 8/12/2012 1:53 pm
Did I mention I downloaded songs from youtube to my harddisk, then watched them / listened to them... from youtube again? When we understand that crazy behavior of mine fully, we'll have got why the cloud takes it all: It fulfills hidden needs of which nobody is conscious yet, and which makes its success, and only a part of which is the factor "others will know your opinions". If I had thought about these things2 1/2 years ago, I would have bought Apples shares in time = when they came out with their first slate; now's too late. But I've come to the intermediate conclusion that the cloud will indeed take fully over, and that's going to have some implications for the phenomenon I described here days ago: So there's an INNER EXPLANATION for cloud taking over and desktop applics vanishing, et because of that inner explanation, this will never be turned over, not even partially so. (Ok, my realization of this comes late, but then, I'm one of few giving reasons.) ENOUGH! Oh, just one: If you don't agree with the simple reasons they give, dig for deeper ones. (Or something like that.) ;-)