Document Management Systems

< Next Topic | Back to topic list | Previous Topic >

Pages: < 1 2 3 4 5 >

Posted by Graham Rhind
Jan 28, 2013 at 08:18 AM

Alexander Deliyannis wrote:
When you say you put files in OneNote, do you mean actually embed them,
>i.e. import them _into_ OneNote notebooks?
> >And are you talking about different kinds of files (which themselves
>alone can be really big), including Word documents, images and PDFs?

It depends how one defines “archive”, right? Data files which I’m not likely to need again get zipped and backed up on external media. But I have a load of customers who contact me with “We met and discussed x in 1998. Remember?”. Er, no. Or “what work did you do for my predecessor in 2002”. Because I have around 130000 document files scattered over my PC I need to put some of them in a place where they can be easily found, previewed and tagged, so that I can get answers to questions like that. So I both link and read any type of file (particularly Office, image and pdf files) into OneNote because that allows me to find the file and to view the file without opening it (as it is “printed” onto a OneNote page), and it allows me to seamlessly sync (properly sync, not use the same file as…) between PCs.

Disk space is not at a premium, so reading a file into a file doesn’t seem as illogical to me (that’s what all these document management programs are doing, plus TreeProjects, UltraRecall etc. etc.). I admit, this is not the ideal way to work - I have a very disciplined and logical brain but a very ill disciplined and illogical workspace. No wonder I need this forum!

Graham

Posted by MadaboutDana
Jan 28, 2013 at 12:15 PM

Having looked into DMS to manage our fairly sophisticated requirements, I heartily concur that you have to define exactly what you want if you’re going to get any value out of a full-scale DMS. There are many purported DMS around, and many of them are vastly overpriced and decidedly underwhelming in actual practice.

We never did find the perfect system. But we have come up with some working procedures that have effectively made a DMS redundant.

First, we have a very controlled file system. It’s not complicated, but all client-related work is stored on a central network server (based on Linux, as a matter of fact), organised by: client, then under each client, project, then under each project, draft documents in separate folders, labelled alphabetically. So our central server structure looks like:

[top level]
/ Client A
/ Client B
/ Client C /
[second level, under e.g. Client C]
// Correspondence
// Project A
// Project B
// Project C /
[third level, under e.g. Project C]
/// a - source docs /
/// b - external drafts /
/// c - internal drafts /
/// d - out to client /
/// e - feedback from client /
/// f - external response /
/// g - internal drafts 2
/// h - out to client 2 etc. etc. etc.

We’ve been using this system for ca. 20 years, and it works very well. Especially when combined with a powerful search engine.

We’ve used various search engines over the years - Windows’ own, then a version of Lucene supplied by IBM. But most recently, we’ve started working with Soonr.com, which means our current working directories can be hosted on multiple PCs, where they are kept synchronised (very efficiently - soonr has the best sync engine I’ve ever encountered, with the possible exception of Dropbox). This also provides a kind of redundancy that acts as a backup.

Stuff that’s more than 6 months old is offloaded to the central server, where it can still be searched (using Lucene).

This in turn means we can use Copernic Desktop Search Pro on individual workstations, so stuff that’s more or less current can be searched at high speed, with full document preview and highlighted search hits, and instant navigation from hit to hit. After much experimentation and dallying with other search engines, I’ve reluctantly concluded that Copernic Desktop Search is the best of the lot - first, because of its flexibility, second, because of its preview and highlighting, third, because of its speed.

Reluctantly, because its support for ‘exotic’ characters (e.g. Arabic, Russian, Chinese) is occasionally iffy.

This simple solution is very low-cost compared to the vast prices charged by enterprise DMS providers. It also pretty much obviates the need for tagging, because Copernic is so quick and convenient, and also because it supports Boolean searches (including date-bounded, subfolders etc.). I have major reservations about tagging in any case (the much-explored ‘what you know you’ve got’ vs. ‘what you don’t know you’ve got’ issue).

Soonr has a reasonably good built-in search engine, which works well on the web and in its iOS client. The latter also has document editors which do a reasonable job on e.g. MS Office files.

But I like the idea of ‘printing’ documents to OneNote, especially the portability aspect. That’s definitely worth exploring, especially in view of the very good OneNote clients now available on other platforms (notably Outline+ on iOS).

We take a similar approach to e-mail, archiving everything in our various e-mail accounts to the outstanding MailStore (http://www.mailstore.com/). The latter is available as a standalone - desktop - program that’s free of charge; but the price for the server is extremely modest. I cannot speak highly enough of MailStore - it’s utterly reliable, archives e-mail fast and efficiently, and has a great built-in search engine (with support for ‘smart search’ folders).

This remarkably simple system means we can generally isolate specific documents relating to specific projects from specific clients for any time over the past 10 years or so (or even further back, in fact), which in turn means that we are often able to help our clients who’ve lost copies of their own documentation (despite their complex in-house enterprise DMS!). All part of offering a value-added service!

Posted by Dr Andus
Jan 28, 2013 at 01:19 PM

MadaboutDana wrote:
>Copernic Desktop Search is the best of the lot - first, because of its
>flexibility, second, because of its preview and highlighting, third,
>because of its speed.

Copernic Home version does occasionally surprise me with not finding files that I know I do have. I do like it overall but due to these blind spots (for which I don’t know the reasons for) I find it necessary to complement it with Everything Search. It’s nice that both are free though, and between the two of them I seem to find everything I need.

Posted by Slartibartfarst
Jan 28, 2013 at 02:38 PM

Posted by Richard06118, Jan 27, 2013 at 04:45 AM:
...I use the free version of Copernic Desktop Search and the Sleuthhound
______________________________
Posted by Dr Andus, Jan 27, 2013 at 11:47 AM:
I also thought that file and folder explorers are a different category from document management software. Probably a lot depends on the volume of existing and incoming documents one needs to manage, and the frequency of having to recall them, which will call for specific solutions.
But re file/folder explorers, I think it’s definitely a category where if you try a Win Explorer replacement, you’ll have few reasons to return to Win Explorer, even with the free versions, such as Explorer2 lite.I’m a very basic user of Directory Opus (just scratching the surface) but since I’ve tried it I couldn’t live without (mostly for reasons of time-saving):
- opening pre-set number of folders (in tabs) simultaneously, when Dopus launches;
- switching on vertical dual-panes with one click, whenever needed;
- having key folders colour-coded (creating a bread-crumb trail to destination folders);
- ability to resit my pre-set default view and location with one click, when things get messy.
- a preview pane that can view pretty much any files out there.
As you said, I didn’t know these features even existed until I tried my first Win Explorer replacement - which is why it’s worth to keep on CRIMPing.
______________________________
Posted by Alexander Deliyannis, Jan 27, 2013 at 03:38 PM:
I will agree that my use of the term Document Management _System_ for Windows Explorer is a bit far fetched. It is first and foremost a File (including Documents) Management _Tool_.
A System implies that there is some method/procedure/framework, as shown by Slartibartfarst in respect to Knowledge Management. Though Windows Explorer has not much to offer in terms of facilitating such work, I maintain that there is some kind of underlying framework, as witnessed by specific conventions:
- Folder tree
- Default user directories
- Shared folders
- Network sharing
- File associations
- Versioning
- One document - one file
- Files can only be in one folder at a time\
...
All of the above are conventions which facilitate document organisation, development, collaboration and distribution. We take them for granted, but they are not; in the thread on Paperport, a tool is mentioned which puts all documents in one big bucket and organises them via tags. There are tools like Scrivener and PageFour which take care of the file infrastructure, while the user works with “books’, “projects’ and “chapters’. There are other tools which store multiple documents in database files, etc.

In short, one can build a Document Management System using Windows Explorer as the Document Management Tool, and standard Windows conventions as the framework to implement specific procedures. There may be more advanced tools, better suited for particular work; the advantage of Windows Explorer is that it’s already there, wherever you may find yourself.
______________________________
Posted by Graham Rhind, Jan 28, 2013 at 08:18 AM:
Alexander Deliyannis wrote:
> When you say you put files in OneNote, do you mean actually embed them, i.e. import them _into_ OneNote notebooks?
> And are you talking about different kinds of files (which themselves alone can be really big), including Word documents, images and PDFs?

Graham
______________________________
—————————————————————————

COMMENTS:
The above and other comments in this discussion thread generally go to illustrate that the de facto Windows file structure can be said to provide a viable and logical DMS (Document Management System) framework - for example, as outlined by @Alexander Deliyannis.
The opening post seemed to indicate that @Gary Carson was interested in a DMS which somehow operated at some kind of (presumably higher) other level - different to the otherwise adequate Windows File System.
I tried to make the point that one’s definition (and criteria for) a DMS would largely be dependent on how one wanted to use the Knowledge incorporated in the documents, and I suggested a definition for KM Knowledge Management. The implication was that if you had such a definition for KM, then your criteria/requirements for a supporting DMS would be more easily identifiable.

@Alexander Deliyannis is arguably spot-on about the existing Windows File System and Windows Explorer. I have also long treated the OS and file management system as a DMS. The frustration for me has been that the OS was never really designed as a DMS but as an OS and associated file management system, so it has shortcomings as far as my peculiar needs go. However, there are various features and functions in the current incarnation of Windows 7 and NTFS that give it much more exciting potential as a DMS. For example (and building on @Alexander Deliyannis’ list):
- A hierarchically rigid folder Tree - each branch is unique and cannot be connected back to the tree at any other point than the origin.
- Built-in visible system default user directories/folders in the Tree.
- Built-in invisible system default user directories in the Tree.
- The ability to add folders to the Tree at almost any point.
- The ability to nest directories.
- The ability to Share directories with other users in the LAN/WAN.
- The ability to set File Associations to single programs that will automatically open/operate on those files.
- Versioning (not sure about how to control this)
- One document

<-->

one file principle.
- The ability to attach metadata, tags, etc. to a file via NTFS alternate data streams (e.g., using the Comments and a hundred or so other fields). However, this to some extent arguably breaches the One document

<-->

one file principle.
- A single unique copy of a File can only exist in one logical folder at a time.
- Logical views of files and folders to create logical flat file “Libraries” that cut across many folders - Microsoft seems to have kludged the reparse point or “junction” concept to enable this.
- There are built-in standard system Libraries.
- The ability to create new user Libraries.
- The ability to create logical reparse point or Junction folders, so that files can seem to be (have the illusion of) existing in more than one logical folder at a time (changing a file changes the file across all views).
- Windows 7 Index/Search:
(a) The OS’ built-in file index/search feature (Search via the Start menu or via Windows Explorer) enables instant full text searching across *all* the file/document types and their metadata, and all the directories that are default and/or that you have additionally defined for the index/search system to operate upon.

(b) The OS’ index/search can include *any text* in single and multi-page .TIF/.TIFF files (which text is automatically OCRed by the index/search feature, if you tell it to do that) and in OneNote Notebooks (databases), but it *excludes* the text in documents embedded as whole files saved within OneNote Notebooks.

(c) The OS’ index/search can include all text in OneNote Notebooks - that includes all text, and any text in any images in OneNote Notebooks (OneNote automatically OCRs any text in images saved in Notebooks, if you enable that facility).

(d) The OS’ index/search can include *Searching for information in audio notes in OneNote*, which includes various audio file types with speeches, songs, and dictation - refer https://www.donationcoder.com/forum/index.php?topic=31755.msg302855#msg302855.
____________________________

CONCLUSIONS:
1. The inbuilt Windows 7 File System, Libraries and Windows Explorer generally seem to provide an adequate and very good DMS with powerful file index/search/find and retrieval functionality. You can build on this with additional search utilities - e.g., Locate32, Copernic, .Everything, etc.
2. Using Microsoft OneNote as a standalone tool - which is integrated with the OS - has the potential to realise some very useful extended DMS and PIM (Personal Information Management) functionality.
3. Using a Windows Explorer replacement (e.g., such as xplorer², which I use) can further extend the DMS and PIM functionality (QED) - e.g., search functionality, flat files, scrap panes, bundle-to-go NTFS files and extended data streams, bulk edit/update of files’ metadata (esp. Comment fields), mass duplicate check/removal, mass folder/tree file comparison.
4. Given the above, and if your database is a client device (desktop PC or laptop) with minimal need for cloud-based storage or collaboration capabilities, then you might *not* need a separate/new DMS.
5. You could gain some very useful DMS functionality by incorporating Reference Management (e.g., Qiqqa) and Library Management (e.g., Calibre) functionality into your solution set. This might carry an overhead/disadvantage of duplication of data.
____________________________

CONSTRAINTS:
1. The 255 character path length max limit seems to still be a limitation in the OS under certain circumstances.
___________________________
2. Using reparse point or Junction folder is risky for the uninitiated. The user could inadvertently/unintentionally wipe out their data, as anything you do to change (e.g., edit or delete) a file actually changes the real file, not some kind of image or copy.
____________________________
3. Reparse points do not exist in entirely happy union with the OS - e.g., backup:
Microsoft support note (KB973455) - http://support.microsoft.com/kb/973455/en-us
“By design, Windows Backup does not traverse reparse points on a computer. However, if a reparse point is directly added to a Windows 7 Library, or if any of the child nodes of a reparse point is directly added to a Windows 7 Library, Windows Backup traverses these locations. This could cause several issues during the backup process.”
____________________________
4. You can share Libraries across LAN/WAN, but may encounter difficulty in handling or sharing reparse points/Junction folders.
**Recommended reading re the latter: XXCOPY TECHNICAL BULLETIN #050 - http://www.xxcopy.com/xxtb_050.htm
____________________________
5. The OS’ index/search capability can apparently include .ZIP files and contents, but does not always work on them for some inexplicable reason.
____________________________

Posted by Foolness
Jan 29, 2013 at 03:03 AM

Gary Carson wrote:
Now that you guys have brought it up, I’ve got to admit that I’m not
>sure why I need an “document management system.” I’m not even sure what
>that is.
> >I don’t work with PDFs very much and I have no need to scan documents.
>All I really need to do is keep the documents and other files that I
>create organized so that I can find them again. But really that’s just a
>matter of exercising some discipline, isn’t it? This is something you
>have to do even if you’re using an actual document management system.
> >The one thing that I’m doing now that I wasn’t doing several years ago
>is storing files that I need to keep long-term in OneNote. That’s my
>long-term archive. I just use Windows Explorer to organize everything
>else and every now and then I’ll do a backup to an external hard disk
>which I keep in my safety deposit box at the bank.
> >I don’t really trust online storage services. It’s a privacy/security
>thing, mostly, and I want to keep control of my own files.

For personal or limited pdf usage, one reason you use DMS is because it’s a form of isolater.

Sometimes folders can work if you just need a separator but other times you need to isolate your work into something that if put on a folder, simply does not function as well. Take index cards if you are not using a software like SuperNoteCard.

Another reason to do this is to have a “profile” of what goes into which usb drive.

Finally you can simply treat it like a to-do list/outliner “with an attachment” because many lightweight or simpler software still don’t do this.

Any type of Management System can be basically honed down to one crucial issue:

What do you define as storage and what do you use to “hoist” documents in that storage?

Google Docs for example is a DMS disguised as an “online word processor”.

...however it’s tough to form that picture if you do not pay attention to the fact that you can hoist it as a collaboration software, as a real time novel writing viewer or as a shareable public webpage that is essentially a better blogging interface on a poorer same themed all white no ads no plugins static webpage.

Pages: < 1 2 3 4 5 >

Back to topic list