A simple guide to using shared drives to capture & classify electronic documents and emails

by Frank 18. July 2014 06:00

I have written previously about ways to solve the shared drives problem (click here) and I have written numerous articles (and a book) about ways to manage emails and electronic/digital records. However, we still receive multiple requests from customers and prospective customers about the best, and simplest, way to effectively manage these problems.

The biggest stumbling block and impediment to progress in most cases is the issue of a suitable taxonomy or classification system. Time and time again I see people putting off the solution while they spend years and tens of thousands or hundreds of thousands of dollars grappling with the construction of a suitable taxonomy. I have written about this topic previously as well and if you want my recommendations please click on this link.

If you really want the simplest, easiest to understand, easiest to use and lowest cost way to solve all of the above problems then please forget about spending the next twelve to eighteen months grappling with the nuances of your classification system. It isn’t necessary.

What you need instead is a natural classification structure that reflects your business processes. Please give your long-suffering end users something they will instantly recognize and can easily work with because it is familiar from their day to day work. Give them something to work with that doesn’t require them to become amateur records managers battling to decipher a complex, hierarchical classification system that requires an intricate knowledge of classification theory to interpret correctly. Give them something that makes it as easy as possible to file everything in the right place first time with absolutely minimal effort. Give them something that makes it as easy as possible to find something.

What I am proposing isn’t a hundred-percent solution and it won’t suit every organization but I guarantee that it will turn chaos into order in any organization that implements it. You may well see it as an eighty-five-percent solution but that is a hell of a lot better than no solution. It is also easy and fast to implement and relatively low cost (you will need some form of RM software).

First up you need to make decisions about what kind of business you are.  Notice that I said “what kind of business you are” not “what kind of records you manage” or “how your business is structured”.  Most importantly, strongly resist the temptation to base your classification structure on your existing business structure or organization’s departments/agencies and instead base it on your most common business processes. Please refer to the following extract from:

Overview of Classification Tools for Records Management by the National Archives of Australia, ISBN 0 642 34499 X (an excellent reference document if you need to understand classification systems).

“Classifying records and business information by functions and activities moves away from traditional classification based on organisational structure or subject. Functions and activities provide a more stable framework for classification than organisational structures that are often subject to change through amalgamation, devolution and decentralisation. The structure of an organisation may change many times, but the functions an organisation carries out usually remain much the same over time.”

I would also strongly resist the temptation to build your classification structure on content; it is way too difficult. Instead, as I have said above, base it on your common business processes.

When I say classification structure I mean the way you name and organize folders in your shared drives. I can’t give you a generic solution because I am not that clever; I don’t know enough about your business. I can however, give you an example.

Please also remember that for the most part, we are dealing with unstructured source information; Word, Excel, PowerPoint, Emails, etc. Emails are a little easier to deal with because they have a limited but common structure, e.g., Date Received, Sender, Recipient, CC and Subject. With other electronic documents we are have far less information and are  usually limited to Author (not reliable), Date Created, Date Modified and Filename. Ergo, as I said earlier, trying to base a classification system on the content of unstructured documents is both difficult and inexact. It is certainly doable but you will have to spend a lot more money on consulting and sophisticated software to achieve your ends.

In my simple example of my simple system I am going to assume that your business is customer (or client) centric, i.e., as opposed to being case-centric or project-centric, etc. The top level of your classification structure therefore will be the client name and/or number. To make it as simple as possible I am going to propose only two levels. The second level represents your most common business processes, that is, what you do with each customer. So for example, I have:

Customer Name

     Correspondence

     Contracts

     Quotes & Proposals.

     Orders

     Incidents

I am also not going to differentiate between emails and other types of electronic documents, I am going to treat them all the same.

Now how does this simple system work?

  1. Staff producing electronic documents don’t have their ‘own’ shared drive, all staff use the common classification structure. This is very important, let one or more people be exceptions and you no longer have a system you can rely on to meet your needs for reliable retrieval and any compliance legislation you are subject to.
  2. Staff drag and drop or ‘save-as’ emails from their email client to the correct sub-folder.
  3. Similarly, staff save (or drag and drop) electronic documents into the correct sub-folder. You can control access if required by applying security to electronic documents.
  4. You purchase or build a document repository (based on any common database such as SQL Server, MySQL, etc.) and within this repository you replicate the folder structure of your shared drives with logical folders and subfolders.
  5. You purchase or build a tool that constantly monitors the shared drives (e.g., using .NET Watcher technology) and that instantly captures a copy of any new or modified document (you do need to configure your repository to automatically version modified documents). You may also decide to automatically delete the original source document after it has been captured.
  6. You build or purchase a records and document management software package that allows you to index, search and report on all the information in your repository.
  7. You train your staff in how to save and search for information (shouldn’t take more than a half to one day) and then you go live.

I would also recommend applying a retention schedule based on sub folder (e.g., contracts) and date created and have the records management system automatically apply it to manage the lifecycle of captured documents. There is no sense in retaining information longer than you have to; it is also a dangerous practice.

Please note that the above is just an example and a very simple one at that. You need to determine the most appropriate folder structure for your organization.

WARNING

Do not let the folder structure become overly complex and unwieldy. If you do, it won’t work and you will end up with lots of stuff either not captured or captured to the wrong place. The basic rules are that if it takes more than few second to decide where to file something then it is too complex and that any structure more than 3 levels deep is too complex.

And finally, this isn’t just a theory, it is something we do in our organization and it is something many of our customers do. If you would like to read more on this approach there are some white papers and more explanations at this link. Alternatively, you can contact us and ask questions at this link.

Good luck.

 

Are you still struggling with physical records management, with paper?

by Frank 16. July 2014 00:01

 

Are you still struggling with physical records management, with paper?

We produced our first computerised records management system in 1984 (when our company was called GMB) and it was called DocFind. It was marketed by the Burroughs Corporation initially to about 100 clients and then we stared marketing DocFind direct and sold it to about another 2,000 clients.

Every one of those clients wanted DocFind just to manage physical records, paper, file folders and archive boxes. There was little or no demand for document imaging and workflow and the term electronic document management had yet to be invented. Office automation was in its infancy. We for example, wrote our letters on an Apple IIe using a word processor called WordStar running under CP/M.

In 1986 we released RecFind, a major remake of the DocFind product. This product was initially marketed by ourselves and NEC and it too focussed just on managing physical records.

However, even in 1986 we knew we had a bigger job to do with the general acceptance of document scanners and workflow so we added imaging and workflow to our product and starting trying to convince our customers and prospective customers to reduce the size of their paper mountain and even to start planning for a ‘Paperless Office’.

In the late 1980s and early 1990s I delivered numerous papers extolling the value of the paperless office and worked hard to convince my customers to make the move to Electronic Document and Records Management (EDRMS).

In the mid-1990s the industry discovered ‘Knowledge Management’ (KMS) and industry consultants lost interest in EDRMS and instead heavily promoted the virtues and benefits of KMS, whatever it was. Maybe this was the time organizations lost interest in eradicating paper as senior IT staff and consultants moved on to more interesting projects like KMS.

In 1995 I delivered my first paper on a totally integrated information management system or what I called at the time the ‘It Does Everything Application’ (IDEA). In 1995 I truly thought the age of physical records management was almost over and that the western world at least would move to fully-automated, paperless processes.

How wrong I was 19 years ago.

Today, despite the advanced functionality of our RecFind 6 Product Suite, almost all of my customers still manage physical records with RecFind 6. At least half of the inquiries that come in via our website are for systems to manage physical records.

There is more paper in the world today than there has ever been and organizations all over the world still struggle with managing paper, vast amounts of paper.

Luckily for us, we never succumbed to the temptation to remove the paper handling features from our products. Instead, we added to them with each subsequent release and redesign/rewrite of RecFind. We had to provide upwards compatibility for our clients as they still managed mountains of paper both onsite and offsite.

Being a little older and wiser now I am never again going to predict the paperless office. I will provide advanced physical records management functionality for my clients as long as they require it.

I haven’t given up the fight but my job is to address the real needs of my customers and they tell me and keep telling me that they need to manager paper, mainly file folders full of paper and archive boxes full of file folders. They need to manage paper onsite in shelving and offsite in warehouses with millions of boxes and we do it all.

We manage paper from creation to destruction and throughout the whole lifecycle. We apply retention schedules and classification systems and we track anything and everything with barcodes and barcode readers. We have enhanced our products to cater for every need and we are now probably responsible for millions of tonnes of paper all over the world.

I still hope for a paperless world but I very much doubt that I am going to see it in my lifetime.

So, if you are still struggling with how to best manage all your physical records please don’t despair, you are most certainly not alone! 

  

What is the future of RecFind? - The Product Road Map

by Frank 19. May 2014 06:00

First a little history. We began in 1984 with our first document management application called DocFind marketed by the then Burroughs Corporation (now called Unisys). In June 1986 we sold the first version of RecFind, a fully-featured electronic records management system and a vast improvement on the DocFind product. Then we progressively added document imaging then electronic document management and workflow and then with RecFind 6 a brand new paradigm and an amalgam of all previous functionality; an Information management system able to run multiple applications concurrently with a complete set of enterprise content management functionality. RecFind 6 is the eighth completely new iteration of the iconic RecFind brand.

RecFind 6 was and is unique in our industry because it was designed to be what was previously called a Rapid Application Development system (RAD) but unlike previous examples, we provided the high level toolset so new applications could be inexpensively ‘configured’ (by using the DRM) not expensively programmed and new application tables and fields easily populated using Xchange. It immediately provided every customer with the ability to change almost anything they needed changed without needing to deal with the vendor (us).  Each customer had the same tools we used to configure multiple applications within a single copy of RecFind 6. RecFind 6 was the first ECM product to truly empower the customer and to release them from the expensive and time consuming process of having to negotiate with the vendor to “make changes and get things done.”

In essence, the future of the RecFind brand can be summarised as more of the same but as an even easier to use and more powerful product. Architecturally, we are moving away from the fat-client model (in our case based on the .NET smart-client paradigm) to the zero-footprint, thin-client model to reduce installation and maintenance costs and to support far more operating system platforms than just Microsoft Windows. The new version 2.6 web-client for instance happily runs on my iPad within the Safari browser and provides me with all the information I need on my customers when I travel or work from home (we use RecFind 6 as our Customer Relationship Management system or CRM). I no longer need a PC at home and nor do I need to carry a heavy laptop through airports.

One of my goals for the remainder of 2014 and 2015 following is to convince my customer base to move to the RecFind 6 web-client from the standard .NET smart-client. This is because the web-client provides tangible, measurable cost benefits and will be the basis for a host of new features as we gradually deprecate the .NET smart-client and expand the functionality of the web-client. We do not believe there is a future for the fat/smart-client paradigm; it has seen its day. Customers are rightfully demanding a zero footprint and the support of an extensive range of operating environments and devices including mobile devices such as smartphones and tablets. Our web-client provides the functionality, mobile device support and convenience they are demanding.

Of course the back-end of the product, the image and data repository, also comes in for major upgrades and improvements. We are sticking with MS SQL Server as our database but will incorporate a host of new features and improvements to better facilitate the handling of ‘big data’. We will continue to research and make improvements to the way we capture, store and retrieve data and because our customer’s databases are now so large (measured in hundreds of Gigabytes), we are making it easier and faster to both backup and audit the repository. The objectives as always are scalability, speed, security and robustness.

We are also adding new functionality to allow the customer to bypass our standard user interface (e.g., the .NET smart-client or web-client) and create their own user interface or presentation layer. The objective is to make it as easy as possible for the customer to create tailored interfaces for each operating unit within their organization. A simple way to think of this functionality is to imagine a single high level tool that lets you quickly and easily create your own screens and dashboards and program to our SDK.

On the add-in product front we will continue to invest in our add-in products such as the Button, the MINI API, the SDK, GEM, RecCapture, the High Speed Scanning Module and the SharePoint Integration Module. Even though the base product RecFind 6 has a full complement of enterprise content management functionality these add-on products provide options requested by our customers. They are generally a way to do things faster and more automatically.

We will continue to provide two approaches for document management; the end-user paradigm (RecFind 6 plus the Button) and the fully automatic capture and classification paradigm (RecFind 6 plus GEM and RecCapture). As has been the case, we also fully expect a lot of our customers to combine both paradigms in a hybrid solution.

The major architectural change is away from the .NET smart-client (fat-client) paradigm to the browser-based thin-client or web-client paradigm. We see this as the future for all application software, unconstrained by the strictures of proprietary operating systems like Microsoft Windows.

As always, our approach, our credo, is that we do all the hard work so you don’t have to. We provide the feature rich, scalable and robust image and data repository and we also provide all of the high level tools so you can configure your applications that access our repository. We also continue to invest in supporting and enhancing all of our products making sure that they have the feature set you require and run in the operating environments you require them to. We invest in the ongoing development of our products to protect your investment in our products. This is our responsibility and our contribution to our ongoing partnership.

 

Is this Microsoft’s worst mistake ever?

by Frank 30. November 2013 06:00

I run a software company called the Knowledgeone Corporation that has been developing application solutions for the Microsoft Windows platform since the very first release of Windows. As always, our latest product offering RecFind 6 version 2.6 has to be tested and certified against the latest release of windows. In this case that means Windows 8.1.

Like most organizations, we waited for the Windows 8.1 release before upgrading our workstations from Windows 7. The only exceptions were our developers workstations because we bought them new PCs with Windows 8 pre-installed.

We are now testing the final builds of RecFind 6 version 2.6 and have found a major problem. The problem is that Microsoft in its infinite wisdom has decided that you can’t install Windows 8.1 over a Windows 7 system and retain your already installed applications.

The only solution is to install Windows 8 first and then upgrade Windows 8 to Windows 8.1. However, if you are running Windows 7 Enterprise this won’t work either and you will be told that you will have reinstall all of your applications.

I am struggling to understand Microsoft’s logic.

Surely Microsoft wants all its customers to upgrade to Windows 8.1? If so, why has it ‘engineered’ the Windows 8.1 upgrade so customers will be discouraged from using it? Does anyone at Microsoft understand how much work and pain is involved in re-installing all your applications?

No, I am not kidding. If you have a PC or many PCs with Windows 7 installed you are going to have to install Windows 8 first in order to maintain all of your currently installed applications. Then, after spending many hours installing Windows 8 (it is not a trivial process) spend more precious time installing Windows 8.1. Microsoft has ensured that you cannot go direct from Windows 7 to Windows 8.1.

Of course, if you are unlucky, you could be living in a country where Microsoft has blocked the downloading of Windows 8, like Australia. Now you are between a rock and a hard place. Microsoft won’t let you install Windows 8 and if you install Windows 8.1 you face days or weeks of frustrating effort trying to re-install all of your existing applications.

 

Here are some quotes from Microsoft:

“You can decide what you want to keep on your PC. You won't be able to keep programs and settings when you upgrade. Be sure to locate your original program installation discs or purchase confirmation emails if you bought programs online. You'll need these to reinstall your programs after you upgrade to Windows 8.1—this includes, for example, Microsoft Office, Apache OpenOffice, and Adobe programs. It's also a good idea to back up your files at this time, too.”

If you're running Windows 7, Windows Vista, or Windows XP, all of your apps will need to be reinstalled using the original installation discs, or purchase confirmation emails if you bought the apps online.”

If the management at Microsoft wanted to ensure the failure of Windows 8.1 they couldn’t have come up with a better plan than the one they have implemented. By making Windows 8.1 so difficult to install they have ensured that its customers will stick with the tried and proven Windows 7 for as long as possible.

Can anyone at Microsoft explain why they thought this was a good idea?

Do you really need a Taxonomy/Classification Scheme with a Records Management System?

by Frank 26. October 2013 06:00

Background

Classification schemes are a way to group or order data; the objective being to group ‘like’ objects together. Classification schemes have been in use for tens of thousands of years, probably beginning when man first realized that there were different types of animals and plants.

We use classifications schemes both to make things easier to find and to add value to a group of objects. By adding value I mean that a classification (describing a group) may provide more information about the members of that group that is obvious from an analysis of a member; this could be referred to as semantics.

Classification schemes are used in all walks of life, for example; in business, in science, in academia and in politics. Are you a liberal or a conservative? Is it a mammal? If it is, is it a marsupial or a monotreme or a placental mammal? This last example illustrates the usual hierarchical arrangement of classification schemes.

In business, we have long used classification schemes to order business documents, that is, records of business transactions. We are all familiar with file folders and filing cabinets; these things are tools of a classification scheme. They make implementing a classification scheme easier as do numbering systems, colors, barcodes and Lektrievers.

With the first commercial availability of mainframe computers in the early 1960s came our first attempts to computerize filing systems. It was also in the 1960s that we saw the first text indexing systems and the first sophisticated search algorithms.

The advent of text indexing and search algorithms allowed us to do a much better job of classifying data but more importantly, they allowed us to do a much better job of finding data.

Let’s not get in a debate about terminology and acronyms

Our industry (information management to use an all-encompassing term) is often its own worst enemy. It creates terms and acronyms at will with both confusing and overlapping definitions. Then it wonders why normal end–users exhibit first bewilderment and then disinterest. Let’s look at a few examples, e.g., RIMS, RMS, DMS, EDRMS, IAMS, CMS, ECM and KMS.

Do you realize that the process of records management is part of each of the preceding acronyms?

For my part I will stick with my old friend the world records management standard, ISO 15489. It tells us that records are evidence of a business transaction and that records are in any form including paper, electronic documents and emails (I know emails are electronic documents but the world generally differentiates them because emails are ‘different’).

So as far as I am concerned the term Records Management System or RMS includes everything we do and is easily recognized and understood so this is the term and acronym I will use in this paper.

Browsing versus searching

Classification systems are very good at making it easier for us to find information by browsing but not very helpful when we are searching.

Most classification systems require you to first ‘browse’ before finding the exact information you want; you usually have to examine multiple objects before you find the one you want. But this is what classifications systems are very good at; because they organize data in a logical (to a human being) way, we usually know where to begin looking. This is why a classification scheme works so well with a manual filing system (multiple cabinets or multiple shelves of file folders)

Classification schemes are great for physical data and, I would say, absolutely necessary for physical data; how else would you organize fifty-thousand file folders (containing seven and a half million pages) in a huge filing room with hundreds of shelves?

However, with computers I don’t need to browse through multiple objects to find the one I want. By using techniques more appropriate to the computer than the filing room, I can search for and find exactly what I want almost instantly. I do not need to leaf through the file folder, I can go directly to the page or directly to the word. I can use the power of the computer.

The following statement will be probably seen as heresy by most practicing records managers but we actually don’t need a classification system (Taxonomy) when computerizing records. We just need a way to index and then search for information.

We need to organize our data so an ordinary end-user can easily find what they need without having to be a trained, professional records manager.

Indexing versus classifying

Now I know my interpretation of these two terms will not thrill everyone but the differentiation is an important part of my hypothesis.

Let’s start by looking at two kinds of books, a reference book and a work of fiction. Both have tables of content (a classification system usually called a TOC) but only one (the reference book) has an index (usually).

The TOC for the reference book is both useful and often used. The TOC for the work of fiction is both not useful and rarely used (readers rarely need more than a bookmark).

The TOC for the reference book is way to organize information into a logical form grouping ‘like’ information together in chapters and sections. A TOC for the work of fiction is just a list of chapters; it serves little or no purpose for the typical ‘end-user’, the reader.

All the reader of a fiction book really needs is two things; a bookmark and a ‘memory’ of the author, title, cover combination so he/she doesn’t accidentally buy it again at the airport bookshop before that dreaded long and boring flight.

The reader of the reference book actually needs both the TOC and the index for browsing (the TOC) and searching (the index).

A work of fiction doesn’t usually have nor need an index because the end-user doesn’t require it. A reference book usually has an index and it is often used to go direct to a page (or pages) and locate something very specific.

Drawing parallels with our broader topic, some information needs both a classification system and an index, some information needs just an index and some doesn’t require either (e.g., works of fiction).

Generally speaking, scientific collections require a classification system (a scientific taxonomy); for example, the study of plant species and the study of animal species (e.g., using a phylogenetic classification system). Scientists simply could not communicate with each other without having a detailed and exact classification system in place. But, most end-users are not scientists; they are just people trying to find the best place to store something and want to find it again with the least amount of effort and pain.

My contention is that we can solve all ‘content management’ and records management needs with a solution based on the application of a sensible, simple and self-evident (read that as easy to use or human-oriented) indexing system plus the required searching capabilities (i.e., covering both Metadata and full text). There is a better way.

What indexing system?

Whenever I consult with customers who are contemplating the capture and organization of data (hopefully into information) I always give the same advice. That is, “When you are thinking about how to index data first think about how you will find it later.” Ask this key question of your end-users, “When you are about to search for information what do you usually know about it?” For example:

  • Do you know the last name?
  • Do you know the first name?
  • Do you know the date of birth?

A good indexing scheme reflects real life usage of the system; it reflects how ordinary humans work and ‘see’ information. Put simply, it indexes the information people will later need to search on. It indexes the information people understand and are comfortable with because it is self-evident.

Indexing Emails

An email is usually described as an unstructured document (the same way a Word or Excel document is described as being ‘unstructured’) but in fact it does have structure. Even better, everyone is familiar with an email’s structure so we have very little to teach end-users; that is, we have a simple and self-evident ‘natural’ set of Metadata items to index.

  1. Date of email
  2. Sender
  3. Recipient
  4. CC
  5. BCC
  6. Subject
  7. Text of the body of the email
  8. Text of any attachments

For any normal end-user trying to find an email this is how they would envision an appropriate search.  They wouldn’t care that the email has been classified down to 6 hierarchies using the world’s most sophisticated Business Classification Scheme (BCS).

Understanding what end-users typically ‘know’ before they do a search determines what elements you have to index. This is the key to implementing a successful indexing system.

The above 8 elements of an email are self-evident insomuch as, “Of course I need to be able to search on the sender or recipient or subject….”

Indexing Electronic Documents

Now let’s look at ordinary electronic documents (i.e., not emails) because they are much less structured. We all know there are ways to add a common structure using features of MS Office like the information dialog box (asking for keywords etc) and templates and smart tags but these things are rarely and inconsistently used.

With shared drives we usually find some form of ‘evolved’ classification system because managing electronic documents in shared drives is akin to managing millions of pieces of paper in tens of thousands of file folders in hundreds of filing cabinets. Unfortunately, the good intentions and purity of design of the original architects of the shared drives folder/sub folder naming conventions (a classification system) are soon corrupted as users make uncoordinated changes and the structure soon becomes unwieldy and incomprehensible.

In my opinion shared drives are OK for the creation of documents (i.e., a work area) but not OK for the management of documents. In fact I would say shared drives are absolutely hopeless for the management of documents as history and practice will attest.

Once again we need an appropriate indexing system and once again we need to ask, “What do people know at the time of the search?” For example:

  1. Original filename
  2. Original path/filename
  3. Type/suffix – e.g., .DOC, .XLS, .PDF, etc
  4. Author
  5. *Subject

Metadata and the Dublin Core

Let me quote from the Dublin Core website:

http://dublincore.org/

“The Dublin Core Metadata Element Set is a vocabulary of fifteen properties for use in resource description. The name "Dublin" is due to its origin at a 1995 invitational workshop in Dublin, Ohio; "core" because its elements are broad and generic, usable for describing a wide range of resources.”

To quote Wikipedia:

http://en.wikipedia.org/wiki/Dublin_Core

“It provides a simple and standardized set of conventions for describing things online in ways that make them easier to find. Dublin Core is widely used to describe digital materials such as video, sound, image, text, and composite media like web pages.”

The Simple Dublin Core Metadata Element Set (DCMES) consists of 15 elements.

  1. Title
  2. Creator
  3. Subject
  4. Description
  5. Publisher
  6. Contributor
  7. Date
  8. Type
  9. Format
  10. Identifier
  11. Source
  12. Language
  13. Relation
  14. Coverage
  15. Rights

To my mind the Dublin Core is an excellent set of elements for describing almost any ‘record’ because it is both simple and appropriate to both computers and ‘normal’ end-users. As a professional, I like the elegance of the Dublin Core.

I also like the basic principle because it fits in with my hypothesis. That is, there is a better way to store, index and find records than a complex and unwieldy Taxonomy.

The Full Solution?

  • We need an application that stores documents of all types, i.e., all types of content.
  • We need an application that indexes both Metadata and full text.
  • We need an application with a customer configurable Metadata model.
  • We need an application that allows you to search on both Metadata and full text in a single search.
  • We need a search that combines BOOLEAN and numeric operators, e.g., AND, OR, NOT, =, <, >, etc.
  • We need a ‘standard’ Metadata definition (a Class if you will) that includes a simple (not more than 20 in my estimation) set of data elements that includes all of the elements necessary to index all of the types of documents (including file folders and paper) that you manage.
  • We need an application that includes all types of data capture, e.g., from the file system, from the native application, from a scanner, etc.
  • We need an application with a comprehensive security system.
  • We need an application with all reporting options, e.g., both standard reports and ad hoc reports.
  • We need an application with a configurable audit trail.
  • We need an application with comprehensive import and export capabilities.

 

The standard Metadata definition (Master Metadata Class)

I have come up with a limited set of elements that I believe can be used to index and find any type of record, paper or electronic. I have borrowed heavily from the Dublin Core because it makes good sense to do so; there is no need to reinvent the wheel.

#

Element

Explanation

1

Title

A name given to the record. Typically, a Title will be a name by which the record is formally known.  Text, e.g., "Business Plan for 2010"

2

Author(s)

The sender or author, E.g., Mark Twain or f.mckenna@k1corp.com

3

Dated

The original date of the document or published date

4

Date Received

Date received by the recipient or recipient's organization, whichever is the earlier

5

Original Name

e.g., filename or file\pathname for electronic documents  - C:\franks stuff\sample.xls

6

Primary Identifier

An unambiguous reference to the record within a given context. E.g., The file number

7

Secondary Identifier

An unambiguous reference to the record within a given secondary context. E.g., The case number or contract number or employee number

8

Barcode

Barcode number or RFID tag

9

Subject

The topic of the record. Typically, the subject will be represented using keywords or key phrases. Recommended best practice is to use a controlled vocabulary.

10

Description

An account of the record. Description may include but is not limited to: an abstract, a table of contents, a graphical representation, or a free-text account of the record.

11

Content

Words or phrases from the text content of the main document and attached documents

12

Contents

Description of contents if the document is a container, e.g., an archive box

13

Recipient(s)

Addressed to, sent to etc. People or organizations.

14

CC recipient(s)

CC and BCC recipients

15

Publisher

An entity responsible for making the record available.  Company or organization that either published the document or that employs the author

16

Type

The nature or genre of the record, usually from a controlled list, e.g., complaint, quotation, submission, application, etc.

17

Format

The file format, physical medium, or dimensions of the record. E.g., Word, Excel, PDF, etc

18

Language

e.g., English, French, Spanish

19

Retention

 The retention code determining the record’s lifecycle

20

Security

Access rights, security code, etc

 

My contention is that by using an ‘index set’ like the above 20 Metadata elements you can index, manage and retrieve any ‘record’ regardless of form and content.

What about all the standards ‘out there’?

There is a plethora of local, state, federal, industry and international standards pertaining to the management of records. Examples are DoD 5015, MoReq2, Dublin Core, ISO 15489, VERS etc and literally thousands of standards for Metadata.

The problem with most of these standards is that they are extraordinarily difficult to read and understand (even the Dublin Core documentation can be heavy going). I would draw a parallel back to the times when the Bible was in Latin but Christians were supposed to order their lives by its teachings. The problem being that only about 0.025% of Christians spoke Latin. Ergo, how do you order your life by a book you can’t read?

My assertion is that most records managers do not fully understand the standards they are charged with enforcing.

The problem isn’t with the records managers; it is with the people who write the standards. The standards are not written for records managers, they are written for academics and technical people (i.e., systems engineers who are experts in XML).  Just like the Latin Bible, they are not written in the language of the intended user.

And even when you do think you have a grasp of the fundamentals there are always multiple points to be clarified (as to the exact meaning) with the standards authority.

What about Retention/Disposal schedules?

This should probably be the subject of another paper because retention schedules have also become way too complex, unwieldy and difficult to understand and apply.

The question will be, “How can I do away with my classification system when my retention codes are linked to it?”

I have looked at hundreds of retention schedules and every single one has been way too complicated for the organization trying to use it. Another problem is that very few of the authorities that compile retention schedules do so with computers in mind. This means that we end up with lots of very vague conditional statements that are almost impossible to computerize.

Most retention schedules are written for archivists to read, not for computers to process. This is the heritage of retention schedules; they assumed an appraisal process by a trained and expert archivist.

The Continuum model or ‘Whole of Life’ model or File Plan model all assume we will allocate a retention code at the time the record is created, not during a later appraisal process. This made much more sense and allowed us to better manage the record throughout its life cycle. However, many such schemes also linked the retention code to a classification term or embedded the retention codes within the classification system. This of course made the classification system even more complex and difficult to understand and apply.

To my mind no organization needs more than ten retention codes (shortest period, longest period and eight in between) and three life cycles (e.g., active, inactive, destroyed). This is also probably heresy to a lot of the records management profession but, I would ask them to think about the proposition that something that was entirely appropriate to the manual world is not necessarily entirely appropriate to the computerized world. There is an easier and simpler way to manage retention and there is no need to embed retention codes into the classification system just as there is no need for a classification system in any modern, computerized records management system.

What about File Folders and Archive Boxes?

This is the classic stumbling block. This is when the records manager tells you that all the standards require you to use the same taxonomy for emails and electronic documents that he/she uses for traditional file folders and archive boxes.

You need to explain that the classification from the manual paper handling world is inappropriate to the computerized world, that it is an anachronism. You need to explain that all it will add is complexity, massive cost, confusion and a seriously negative attitude to end-users. You should say it is time to discard techniques and tools from the eighteenth century and adopt techniques from the twenty-first century. You should say you have a much better way. Then you should probably duck and run. Failing all else, blame me and give them my email address.

 

 

Records Management in the 21st century; you have computers now, do it differently

by Frank 1. June 2013 06:32

I own and run a computer software company called the Knowledgeone Corporation and we have specialised in what is now known as enterprise content management software since 1984 when we released our first product DocFind. We are now into the 8th iteration of our core and iconic product RecFind and have sold and installed thousands of RecFind sites where we manage corporate records and electronic documents.

I have personally worked with hundreds of customers to ensure that we understand and meet their requirements and I have also designed and specified every product we have delivered over the last 29 years so while I have never been a practicing records manager, I do know a great deal about records and document management and the vagaries of the practise all around the world.

My major lament is that many records managers today still want to run their ‘business’ in exactly the same way it was run 30 or 50 or even a hundred years ago. That is, as a physical model even when using computers and automated solutions like our product RecFind 6. This means we still see overly complicated classification systems and overcomplicated file numbering systems and overcomplicated manual processes for the capture and classification of paper, document images, electronic documents and emails.

It is a mindset that is locked in the past and can’t see beyond the confines of the file room.

I also still meet records managers that believe each and every employee has a responsibility to ‘become’ a junior records manager and both fully comprehend and religiously follow all of the old-fashioned and hopelessly overcomplicated and time-consuming processes laid out for the orderly capture of corporate documents.

I have news for all those locked-in-the-past records managers. Your approach hasn’t worked in the last 30 years and it certainly will not work in the future.

Smart people don’t buy sophisticated computer hardware and application software and then try to replicate the physical model for little or no benefit. Smart people look at what a computer system can do as opposed to 20,000 linear feet of filing shelves or 40 Compactuses and 30 boxes of filing cards and immediately realize that they have the power to do everything differently, faster, most efficiently and infinitely smarter.  They also realize that there is no need to overburden already busy end users by a forcing them to become very bad and very inconsistent junior records managers. End users are not hired to be records managers they are hired to be engineers, sales people, accountants, PAs, etc., and most already have 8 hours of work a day without you imposing more on them.

There is always a better way and the best way is to roll out a records and document and email management system that does not require your end users to become very bad and inconsistent junior records managers. This way it may even have a chance of actually working.

Please throw that old physical model away. It has never worked well when applied to computerised records, document and email management and it never will. Remember that famous adage, “The definition of insanity is to keep doing the same thing and to expect the results to be different”?

I guarantee two things:

1.     Your software vendor’s consultant is more than happy to offer advice and guidance; and

2.     He/she has probably worked in significantly more records management environments than you have and has a much broader range of experience than you do.

It doesn’t hurt to ask for advice and it doesn’t hurt to listen.

I am willing to bet that you are still not managing your emails effectively

by Frank 25. November 2012 06:00

According to various industry surveys, 65% to 75% of companies still have no systems in place to manage email records. Based on my own observations and dialog with Knowledgeone Corporation’s customers and prospects, I would say the percentage is far higher; say 85% or more. My guess is that the industry surveys inadvertently included a number of email ‘cleaning’ systems as email management systems; thereby skewing the figures.

 

Given that there is now a variety of proven email management systems (like Knowledgeone Corporation’s GEM) available for most email servers (e.g., Exchange, GroupWise and Notes) and given the enormous danger of unmanaged email it is, on the surface, difficult to explain the apparent reluctance of organizations to implement email management policies and systems.

 

My own experience leads me to believe that the following are the major reasons organizations do not take this critical step:

1. Lack of ownership and leadership

Email management transects all of the traditional vertical organizational boundaries. There may well be an IT person in charge of the email servers but there is rarely a senior management person in charge of email organization-wide. That is, no one person actually ‘owns’ the problem and no one person has the authority to implement an organization-wide solution.

2. Lack of an understanding of the problem and of the solution

Most of the people who are senior enough in an organization to be aware of this problem do not comprehend the complexities of the problem. They have dialogs with IT people who explain the issues in technical terms, not in business or risk-management terms. Email management should come under an organization’s risk management regime because that is where a great deal of risk lies.

3. Lack of desire to solve the problem plus active opposition to a solution

There are a large number of IT people and others in every organization who simply do not want their emails managed, analysed, scrutinized, indexed and saved. This fact is never going to change and must always be addressed at a senior level by the person responsible for risk management policies and practice. Uncooperative and/or recalcitrant employees should not be allowed to put an organization at risk no matter what their position in the management hierarchy.

4. Confusion over what is involved in complying with a plethora of laws and regulations

One hundred percent of what well-meaning bureaucrats and politicians have done to ‘solve’ what they see as email privacy issues has been badly thought out, badly drafted and counterproductive; simply ill-informed, knee-jerk reactions. As you can see, I am no fan of politicians and bureaucrats who pass knee-jerk laws without understanding or caring about the full implications.

 

As far as I am concerned the privacy issue is secondary to the fact that every employer has to right to determine how its resources are used. Every employer has the right to protect itself. Every employer has the right to tell its employees if private emails are allowed or not. Every employer has the right to tell its employees what is acceptable and what is not acceptable in an email.

 

Solving the so called privacy policy is dead easy; herewith is the McKenna solution.

 

Tell employees that:

1. Private emails are not allowed and all emails will be scrutinized for inappropriate content; or

2. Private emails are allowed (in moderation) but that all emails, including private emails, will be scrutinized for inappropriate content; or

3. Private emails are allowed (in moderation) but that they MUST be identified by the keyword “Private” (or a word or phrase of your choice) in the subject line. All emails without the keyword “Private” in the subject line will be scrutinized for inappropriate content.

5. Confusing and misleading claims by companies marketing email management systems

It is a complex problem (have you ever tried to set up a multi-server email system in a large organization?) often poorly understood and poorly explained by the sales person. Add to this the fact that the sales person is usually speaking to the IT person (who lives in a different universe) who then has to ‘translate’ what he thinks the sales person said to senior management. Too often, the harried sales person, under intense pressure from the IT interrogator, will simply say “Yes” without really understanding the question or its implications.

 

My best advice to senior management is that if they don’t fully understand, keep asking questions until they do or, seek assistance from an independent authority. It is just plain dumb and dangerous to sign something off you don’t really understand.

6. Multiple and conflicting objectives

Is your objective to simply be aware of everything that is in your email store or is it to also meet a plethora of complex and competing regulations and certification standards?

 

Have you inadvertently set the goal post too high? Have you made the problem many times more complex than it should be? Has it become a “Wish List” instead of a requirement? Is the selection of a suitable product always held up by someone demanding that it has to also do something else? Has your horse now morphed into a camel?

 

My best advice? Why don’t you try ‘Getting wet slowly’ and review your needs again when the basic but critical email management problem is solved?

 

In the end it is about ownership, understanding and will. If just one senior person with the necessary authority understands the problem and commits to a solution then it will happen. The solutions are out there; they are just waiting for a committed purchaser with a clear and simple view of what needs to be achieved.

 

You must be aware of what is in your email store and you must be alerted to infringements before they grow into expensive problems. You can’t do this without an email management system in place.

 

Do you really need all those boxes of records in offsite storage?

by Frank 11. November 2012 06:39

Is it jobs or useless paper records?

It is my belief that all over the western world companies and government agencies are wasting enormous amounts of money maintaining boxes of paper on the dusty but lucrative shelves of offsite storage companies like Grace Records Management, Iron Mountain and Crown Records Management. In total, it must be hundreds of millions (I know of one Australian company that spends a million dollars a year on offsite storage at multiple offsite repositories and doesn’t even know what its holdings are) or even billions of dollars a year; most of it wasted.

It is almost enough for me to dive into debt to build an offsite storage facility and then buy a few vans and shredders. I say almost because I am not a hypocrite and I wouldn’t be able to sell a service to my customers I didn’t believe in. For the life of me, I cannot understand why senior management delegates this level of expenditure to junior or mid-level managers when it really should be scrutinized at board level like every other significant cost.

Even the advent of the Global Financial Crisis (GFC) beginning in 2008 doesn’t seem to have woken up senior management or board members to this area of massive waste. Instead, big corporations and government are ‘saving money’ laying off staff and outsourcing jobs to third world and developing countries. Where is the sense in that when there are easier and less disruptive and more ‘humane’ savings to be made by simply reducing the money being paid to store useless paper records that will never be referenced again? How would you feel if management laid you off because they thought it was more important to keep paying for boxes of old paper they will never use again?

Is it really only me that sees the unfairness and absurdity in this archaic paradigm? Why is the huge cost of the offsite storage of useless paper often overlooked when management is fighting to find cost savings? Why are people’s livelihoods sacrificed in deference to the need to maintain old, never-to-be-referenced-again, useless paper? Is it just because senior management is too busy with more important stuff like negotiating their next executive pay increase?

If you talk to the records manager you will be told that all that paper has to be maintained whatever the cost because of the Retention Schedule. In most cases, the Retention Schedule will be mentioned in the same way one talks about the Bible. That is, it is holy and sacrosanct and anyone who dares question it will be charged with heresy and subjected to torture and extreme deprivation in a rat infested, mouldy, dark and damp cell in the basement.

But, dig deeper and you will discover that the Retention Schedule is way too complex for the organization. You will also discover that no one really understands or can explain all the variations and that the application of it is at best, haphazard and irregular. This is when you will also discover that no one in records can actually justify why a huge percentage of those old, dusty and now irrelevant paper records are still costing you real hard cash each and every month. More importantly, they may have also cost you some of your most trusted and most valuable employees.

Isn’t it time someone senior actually looked at the money you are spending to manage mostly paper rubbish in very expensive containers?

Could you manage all of your records with a mobile device?

by Frank 2. September 2012 06:00

I run a software company and I design and build an enterprise strength content management system called RecFind 6 which among other things, handles all the needs of physical records management.

This is fine if I have a big corporate or government customer because the cost is appropriate to the scale of the task at hand. However it isn’t fine when we receive lots of inquiries from much smaller organizations like small law forms that need a records management solution but only have a very small budget.

A very recent inquiry from a small but successful engineering company was also a problem because they didn’t have any IT infrastructure. They had no servers and used Google email. However, they still had a physical records management problem as well as an electronic document management problem but our solution was way outside of the ballpark.

Like any businessman I don’t like to see business walk away especially after we have spent valuable consultancy time helping the customer to understand the problem and define the need.

We have had a lot of similar inquiries lately and it has started me thinking about the need for a new type of product for small business, one that doesn’t require the overhead and expense of an enterprise-grade solution. It should also be one that doesn’t require in-house servers and a high overhead and maintenance cost.

Given our recent experience building a couple of iOS (for the iPhone and iPad) and Android (for any Android phone or tablet) apps I am of the opinion that any low cost but technically clever and easy-to-use solution should be based around a mobile device like a smart phone or tablet.

The lack of an in-house server wouldn’t be a problem because we would host the solution servers at a data centre in each country we operate in. Programming it wouldn’t be a problem because that is what we do and we already have a web services API as the foundation.

The only challenge I see is the need to get really creative about the functionality and the user interface. There is no way I can implement all the advanced functionality of the full RecFind 6 product on a mobile device and there is no way I can re-use the user interface from either the RecFind 6 smart-client or web-client. Even scaled down the user interface would be unsuitable for a mobile device; it needs a complete redesign. It isn’t just a matter of adapting to different form factors (screen sizes), it is about using the mobile device in the most appropriate way. It is about designing a product that leverages off the unique capabilities of a mobile device, not trying to force fit an application designed for Windows.

The good news is that there is some amazing technology now available for mobile devices that could easily be put to use for commercial business purposes even though a lot of it was designed for light weight applications and games. Three examples of very clever new software for mobile devices are Gimbal Context Aware, Titanium Mobile SDK and Vuforia Augmented Reality. But, these three development products are just the tip of the iceberg; there is literally a plethora of clever development tools and new products both in the market and coming to market in the near future.

As a developer, right now the Android platform looks to be my target. This is mainly because of the amount of software being developed for Android and because of the open nature of Android. It allows me to do far more than Apple allows me to do on its sandboxed iOS operating system.

Android also makes it far easier for me to distribute and support my solutions. I love iOS but Apple is just a little too anal and controlling to suit my needs. For example, I require free access to the file system and Apple doesn’t allow that. Nor does it give me the freedom I need to be able to attach devices my customers will need; no standard USB port is a huge pain for application developers.

I am sorry that I don’t have a solution for my smaller customers yet but I have made the decision to do the research and build some prototypes. RecFind 6 will be the back-end residing on a hosted server (in the ‘Cloud’) because it has a superset of the functionality required for my new mobile app. It is also the perfect development environment because the RecFind 6 Web Services SDK makes it easy for me to build apps for any mobile operating system.

So, I already have the backend functionality, the industrial-strength and scalable relational database and the Web Services API plus expertise in Android development using Eclipse and Java. Now all I have to do to produce my innovative new mobile app is find the most appropriate software and development platforms and then get creative.

It is the getting creative bit that is the real challenge. Wish me luck and watch this space.

 

Is Information Management now back in focus?

by Frank 12. August 2012 06:00

When we were all learning about what used to be called Data Processing we also learned about the hierarchy or transformation of information. That is, “data to information to knowledge to wisdom.”

Unfortunately, as information management is part of what we call the Information Technology industry (IT) we as a group are never satisfied with simple self-explanatory terms. Because of this age-old flaw we continue to invent and hype new terms like Knowledge Management and Enterprise Content Management most of which are so vague and ill-defined as to be virtually meaningless but nevertheless, provide great scope for marketing hype and consultants’ income.

Because of the ongoing creation of new terminology and the accompanying acronyms we have managed to confuse almost everyone. Personally I have always favoured the term ‘information management’ because it tells it like it is and it needs little further explanation. In the parlance of the common man it is an “old un, but a good un.”

The thing I most disliked about the muddy knowledge management term was the claim that computers and software could produce knowledge. That may well come in the age of cyborgs and true artificial intelligence but I haven’t seen it yet. At best, computers and software produce information which human beings can convert to knowledge via a unique human cognitive process.

I am fortunate in that I have been designing and programming information management solutions for a very long time so I have witnessed first-hand the enormous improvements in technology and tools that have occurred over time. Basically this means I am able to design and build an infinitely better information management solution today that I could have twenty-nine years ago when I started this business.  For example, the current product RecFind 6 is a much better, more flexible, more feature rich and more scalable product than the previous K1 product and it in turn was an infinitely better product than the previous one called RecFind 5.

One of the main factors in them being better products than their predecessors is that each time we started afresh with the latest technology; we didn’t build on the old product, we discarded it completely and started anew. As a general rule of thumb I believe that software developers need to do this around a five year cycle. Going past the five year life cycle inevitably means you end up compromising the design because of the need to support old technology. You are carrying ‘baggage’ and it is synonymous with trying to run the marathon with a hundred pound (45 Kg) backpack.

I recently re-read an old 1995 white paper I wrote on the future of information management software which I titled “Document Management, Records Management, Image Management Workflow Management...What? – The I.D.E.A”. I realised after reading this old paper that it is only now that I am getting close to achieving my lofty ambitions as espoused in the early paper. It is only now that I have access to the technology required to achieve my design ambitions. In fact I now believe that despite its 1995 heritage this is a paper every aspiring information management solution creator should reference because we are all still trying to achieve the ideal ‘It Does Everything Application’ (but remember that it was my I.D.E.A. first).

Of course, if you are involved in software development then you realise that your job is never done. There are always new features to add and there are always new releases of products like Windows and SQL server to test and certify against and there are always new releases of development tools like Visual Studio and HTML5 to learn and start using.

You also realise that software development is probably the dumbest business in the world to be part of with the exception of drug development, the only other business I can think of which has a longer timeframe between beginning R&D and earning a dollar. We typically spend millions of dollars and two to three years to bring a brand new product to market. Luckily, we still have the existing product to sell and fund the R&D. Start-ups however, don’t have this option and must rely on mortgaging the house or generous friends and relatives or venture capital companies to fund the initial development cycle.

Whatever the source of funding, from my experience it takes a brave man or woman to enter into a process where the first few years are all cost and no revenue. You have to believe in your vision, your dream and you have to be prepared for hard times and compromises and failed partnerships. Software development is not for the faint hearted.

When I wrote that white paper on the I.D.E.A. (the It Does Every Thing Application or, my ‘idea’ or vision at that time) I really thought that I was going to build it in the next few years, I didn’t think it would take another fifteen years. Of course, I am now working on the next release of RecFind so it is actually more than fifteen years.

Happily, I now market RecFind 6 as an information management solution because information management is definitely back in vogue. Hopefully, everyone understands what it means. If they don’t, I guess that I will just have to write more white papers and Blogs.

Month List