Evolution of Enterprise Content Management

By: Brian J. Stewart

The volume of content that enterprises produce and consume on an annual basis is growing exponentially. This growth is primarily driven by increased globalization and regulatory compliance. It is also fueled by the newer marketing and sales channels created over the past few decades including social media, websites, and mobile applications.

There are also several significant technology advancements during the past two decades that dramatically changed how enterprises manage content and documents. This includes Enterprise Document Management Solutions (EDMS), Cloud Computing, and Information Storage.

This article will provide historical context on enterprise document management, as well as a framework to understand next generation enterprise document management. It is important to understand the history of document management in the enterprise, as well as next generational trends and drivers that will drive enterprise document management in the future. This will enable organizations to develop a better strategy to managing content, as well as eliminate the many constraints and pain points with legacy document management solutions (including many ‘Modern EDMS Solutions’).

Let’s start with a brief overview of the types of content typically managed by an enterprise. Then review the pre-EDMS era and its challenges, EDMS era and its challenges. Then we will review EDMS.Next, the next generation EDMS era, and review the business drivers which will all determine the development an effective strategy for content management.

Types of Documents

Forrester classifies content that organizations create into three main categories: Business, Persuasive, and Transactional. Business content refers to content created by organizations through normal daily operations, such as documents and presentations. Persuasive content refers to content designed to influence or persuade customers, such as marketing materials and websites. Transactional content refers to content created by partners and customers through the execution of business transactions, such as purchases or sales. [1] The following diagram shows many examples of content created by different departments

 

Next let’s look at how businesses managed content through the different phases of enterprise content management.

Pre-EDMS Era

In the era of computing prior to the development of EDMS solutions, organizations typically stored documents in network file shares. The following diagram depicts the typical document management solution for most organizations:

Content was created and consumed on personal computers which were networked together via a LAN/WAN. Authors, reviewers, and approvers were given read/write access to the network shares, while viewers were given only read-only access. Security was controlled by Network Administrators utilizing the server operating systems and Directory Services.

Challenges with this era of document management

Although this solution worked, it had several limitations including:

No way to maintain versions of documents without complex directory structures
No way to facilitate the review and approval of documents, requiring documents to be printed, manually routed for review and approval, signed (wet signatures), and scanned for retention
No way to manage metadata or further classify documents outside of a complex directory structure or complex file names
Difficulty searching and locating content

Storing documents in network files shares are not without benefits. They are quick and easy to setup, require little technical support, and can be self-managed by users. Additional storage can easily be added through additional hard drives or Network-Attached Storage (NAS). Business owners of content can easily reclassify and reorganize content by changing directory structure, file names, or self-managed databases and spreadsheets. Most organizations, even very large corporations, still have significant remnants of this era with documents continuing to be stored on network shares.

EDMS Era

The next technological phase for managing documents was the arrival of Electronic Document Management System (EDMS). EDMS is a system which manages the entire lifecycle of documents including creation, review, approval, and retiring of documents. These solutions also provide integrated version management, metadata management, automated review and approval through workflow, and indexing and searching content. EDMS promised many things and largely delivered on those promises. The following diagram shows the typical EDMS architecture:

Changes and Similarities with Previous Era of Document Management

The consumers and producers of content were still largely internal authors, reviewers, approvers, and viewers. These users still accessed documents through traditional laptop and desktop computers, however users were able to better organize, manage, and locate content.

Over the years, EDMS vendors added more and more functionality and complimentary services, such as:

Indexing and Document Search: Full-text indexing to facilitate locating documents based specific keywords in content and metadata search to facilitate locating documents based on metadata
Document History Audit Trail: Electronic auditing of document actions to facilitate internal and external audits
Security and Access Management: Complex security management based lifecycle state, user role, and workflow status
Version Management: Maintaining of version history including major and minor versions
Lifecycle Management: Maintaining of state of document (i.e. Draft, In-Review, Approved, Obsolete)
Metadata Management: Integrated metadata or properties including enforcement of business rules
Workflow Services: Electronic review and approval workflows including Annotation Services to facilitate document review
Electronic Signatures: Electronic signatures to replace manually signing paper documents
Library Services: Management of documents including check-out, locking, and check-in
Capture Services: Capture Integrations to enable the automated creation of content through scanners

Impact of These Changes

EDMS is not without many shortcomings and challenges. Primarily due to familiarity, many users still preferred the traditional browse or drilling down to locate documents in EDMS. For this reason many EDMS implementations continued to mimic foldering of documents in complex directory structures. In addition, users continued to use ‘meaningful file names’.

In addition, EDMS created the opportunity to manage security at a more granular level, as security was now managed by Application Administrators rather than Network Administrators. This often resulted in overly complicated security models. Excessively complex security models are difficult to test, adversely affect application performance, and are subject to the need for numerous exceptions.

Another challenge with EDMS implementations is also one of its key benefits – metadata management. Although metadata management was a critical and driving element of EDMS implementations, many problems surfaced. Metadata management either relies on free-text fields which enable users to enter any text or selection lists which enables users to select from predefined fields. The following are the key challenges and problems encountered with many EDMS implementations:

Lack of metadata consistency for free text fields
Need to maintain data dictionaries in each system with little consistency across systems
Data redundancy across systems
Inconsistency of metadata and taxonomies across made system integrations difficult, expensive, and fragile
Creation of data silos where content was locked in repositories due to taxonomy and metadata inconsistencies, complex security models, as well proprietary nature of typical enterprise systems and lack of standards and open interfaces
Discoverability and repurposing of content is difficult
The biggest challenge, however, was the creation of data silos and high costs for enterprise application integrations. Organizations were forced to spend significant capital expenditures to build interfaces in an attempt to breakdown these content silos. These integrations are often fragile and sensitive to any environment or technology changes and thus often require significant operating expenditures to support. Many organizations are stuck in this era.

Next Generation of Content Management

The next phase of enterprise document management is Next Generation EDMS or what will be referred to as EDMS.Next in this article. EDMS.Next brings about several changes in terms of technology and process. The following diagram shows the EDMS.Next:

The creation and consumption of content is now seldom confined to enterprise boundaries, as content is created and consumed by suppliers, partners, customers, vendors, and regulatory authorities.
The content creation and consumption is also dramatically changed within the enterprise, as creation and consumption is no longer constrained to traditional application boundaries. Metadata is commonly managed across many business processes and data systems.
Content producers and viewers are also accessing content from an increasing number of devices in addition to traditional personal computers, such as smartphones and tablets.
As a result of globalization, content creation and consumption must consider geographic and language differences and global application performance
The hyper-competitive environment requires organizations to continually improve user productivity and operational efficiencies, as well as reduce costs

There are also several technological advancements that impact managing content in the enterprise, including the creation and maturation of Business Process Management (BPM), knowledge management, enterprise portals, cloud computing, and web services for Enterprise Application Integration (EAI). In addition, the maturation of the Internet and the creation of an always connected world is dramatically changing content management.

EDMS vendors added more and more functionality and complimentary services, such as:

Content Creation: Dynamic generation of content based on data
Content Archival: Archiving of content that is no longer needed on a daily basis
Notification Services: Subscriptions and notifications including notifying users of document events such as creation, review, approval, and obsoleting content
Content and Metadata Publishing: Automatic publishing of content and metadata to target systems and repositories based on business rules
Collaboration Services: Development of content is increasingly becoming a collaborative effort and EDMS is evolving to include more collaborative features such as concurrent real-time editing

Impact of These Changes

This requires a dramatically different approach to vocabulary and taxonomy management, as systems need to use the same vocabulary for effective business process automation and integration. Vocabulary management refers to the management of data dictionary values for metadata selections lists. This has brought about the development of Master Data Management (MDM) strategies to centralize the creation and maintenance of dictionaries within an organization. The data can either be maintained by Data Administrators within an enterprise or data feeds from external data standard organizations. MDM is certainly important, but an equally important, but often missing component of enterprise management strategies is what will be referred to as Enterprise Taxonomy Management (ETM) in this article. ETM ensures that in addition to a common vocabulary, a holistic and strategic thinking is brought to the creation and development of taxonomies within an enterprise. It is important that taxonomy development move beyond the traditional project or application development activity to a key strategic architectural solution similar to MDM.

Key Business Drivers

Next let’s look at the key business drivers for these trends and changes. At a high-level, there is a strong trend to streamline business processes, increase user productivity, improve information accuracy, expand information leverage and accessibility, ensure regulatory compliance, and reduce operational and capital expenditures. There are many business drivers influencing EDMS.Next and it is critical organizations identify and understand these trends in order to develop an effective strategy, rather than an individualized reactive strategy. The below table summarizes these trends and business drivers:

 

Business Driver Description
Government Regulations and Compliance

 

At a global level there is a wave of increased government regulations in order to protect financial systems, healthcare, and citizens. Businesses, especially those in the life sciences and financial industries, must maintain an increasing volume of documentation to ensure compliance with the growing set of regulations and standards.
Standard Consortiums (Oasis, W3C) Organizations are forming working groups to standardize on documentation, terminology, schema, and vocabulary in order to reduce compliance costs and ensure industry consistency. EDMS implementations must adopt these standards to remain competitive and compliant. Examples of standards include:

  • eTMF and eCTD for life sciences
  • DocBook for publications and books
  • Darwin Information Typing Architecture (DITA) for modular based articles
  • OpenDocument and OpenXML for office based documents
  • Universal Business Language (UBL) for standard businsess documents such as invoices, purchase orders
Technology Consolidation Strategy Information Technology departments are facing an increasing number of enterprise applications which are often expensive to develop and maintain. This is especially true with the vast number of EDMS implementations that an organization must support. Many organizations are looking to consolidate EDMS implementations in an attempt to cut costs.
Globalization For good or for bad, globalization is here to stay. Organizations face many challenges with creation and consumption of content across geographic and language barriers. EDMS solutions must adapt to meet the business and performance needs of a global user base.
Data Harmonization Strategy A common vocabulary is critical in order to breakdown content silos and automate an increasing number of business processes, and integrate with EDMS solutions.
Enterprise Portal Strategy Organizations are also looking to improve user productivity by improving information accessibility in much the same way that the Internet has improved information accessibility for the masses – though increasingly smarter search engines. EDMS content and metadata need to be accessible to these portals.
Enterprise Application Integration Strategy Organizations are also increasingly looking for ways to improve user productivity and data accuracy by eliminating redundant data entry and data duplication. This is often accomplished by integrating both EDMS and non-EDMS solutions.
Content and Metadata Repurposing Strategy Organizations are looking to repurpose and reuse content and metadata in different systems and for the information to be tailored for different consumers.
Business Process Automation The creation and maturation of BPM software has led to an increased focus on automating business processes in order to improve efficiencies and effectively compete in a hyper-competitive business climate. The creation and consumption of metadata often involves the automation of business processes across EDMS and non-EDMS solutions.
Mergers and Acquisitions In order to effectively compete in a global economy and ensure compliance with the growing number of government regulations, organizations are increasingly looking to mergers and acquisitions. This consolidation enables organizations to achieve better economies of scale and share and leverage investments across broader revenue producing products and services. This impacts the EDMS strategy in two ways. First, organizations must consolidate technology solutions acquired through mergers and acquisitions. Second, the increasing level of corporate consolidation drives the need for data harmonization.
Interoperability and Information Exchange The creation and maturation of information exchange and storage formats (such as XML, JSON, OpenXML, OpenDocument), as well as data exchange mechanisms (such as web services and data service buses) allows for better information interoperability. EDMS solutions must fully leverage these formats and mechanisms for content and metadata creation and consumption.
Integration with Suppliers and Vendors Organizations are increasingly integrating with suppliers and vendors to improve cost and operating efficiencies. This requires EDMS solutions be more accessible to external users, such as suppliers and vendors. This has implications for the EDMS security models, user interfaces, and service interfaces
Knowledge Management Strategy Sustained competitive advantage is brought about by empowered knowledge workers. An effective Knowledge Management Strategy requires organizations to put the right information in the right people’s hands at the right time. This is accomplished by unlocking information in content repositories, as well as increasing the accessibility of information.
Usability and Common User Experience Strategy Organizations are constantly looking to improve user productivity due the hyper-competitive business climate that that they face. A key contributing element to user productivity is usability and user experience. Unfortunately most EDMS solutions are built on antiquated, clumsy, inflexible and proprietary user interfaces. In addition, each EDMS has its own unique user interface which adversely impacts productivity and raises training and support costs. EDMS solutions need to shift to common task or activity driven user interfaces that have a strong focus on user experience design.
Integration with Strategic Partners Many organizations are turning to strategic partnerships to share costs and leverage the unique capabilities of each organization to jointly develop new products and services. This is especially true in the life sciences industry where smaller firms partner with larger pharmaceutical companies to bring new drugs to market. Similar to integration with suppliers and vendors, this requires a different approach to security, user interfaces, and service interfaces for EDMS solutions.
Real-time Collaboration and Mobility Workers are increasingly ‘connected’ with smartphones, tablets, and wearables. Workers expect information in EDMS solutions to be accessible regardless of device, web browser, or application. Organizations need to rethink their user interfaces and services interfaces to support real-time collaboration and mobility.

Developing an Effective Strategy for Content Management

In order to develop an effective and comprehensive strategy for content management, organizations must understand the evolution of content management, the types of content being produced and consumed within an enterprise, the key advantages and disadvantages of traditional EDMS, and EDMS.Next, which is the next phase in the evolution of content management. Understanding the key business drivers that are resulting in a growing number of challenges for content management solutions will enable organizations to rethink their strategy. Organizations that are constantly ‘putting out fires’ or continually employing knee-jerk responses to each and every new ‘unique’ requirement will miss the big picture – EDMS is changing and solutions need to change too!

Content management strategies need to shift EDMS and information architecture from a solution driven design activity to an enterprise driven design activity. This will lead to next generational solutions that are more flexible, open, and consistent. It will lead to solutions built on standards and a common taxonomy and vocabulary with built-in support for data exchange and information interoperability. The end result of EDMS.Next will:

  • Enable knowledge workers  to produce and consume content regardless of geographic location,  device, or organization
  • Enable systems to communicate regardless of operating systems or platform and across network and organizational boundaries
  • Improve information accuracy and information accessibility while reducing data entry and redundancy

Related Article(s)

  1. Avoiding Common Mistakes with Document Management
  2. Developing an Effective Taxonomy for Document Management Systems
  3. Do’s and Don’ts in Document Management Taxonomy Design
  4. Why Centralized Repositories Aren’t Always a Good Strategy

References

  1. Stephen Powers, Alan Weintraub with Matthew Brown, Anjali Yakkundi. “Plan Your ECM Strategy For Business, Persuasive, Transactional, And Foundational Needs.” Forrester. April 14, 2011. https://www.forrester.com/Plan+Your+ECM+Strategy+For+Business+Persuasive+Transactional+And+Foundational+Needs/fulltext/-/E-res58949.