Open Framework, Information Management Strategy & Collaborative Governance | Data & Social Methodology - MIKE2.0 Methodology
Wiki Home
Collapse Expand Close

Members
Collapse Expand Close

To join, please contact us.

Improve MIKE 2.0
Collapse Expand Close
Need somewhere to start? How about the most wanted pages; or the pages we know need more work; or even the stub that somebody else has started, but hasn't been able to finish. Or create a ticket for any issues you have found.

Infrastructure Management Process Design

From MIKE2.0 Methodology

Share/Save/Bookmark
Jump to: navigation, search

Activities in Phase 4
Phase 4 - Design Increment
Content Model Relationship

Contents

Activity: Infrastructure Management Process Design

Objective. The Infrastructure Management Process Design activity provides a set of requirements for the physical implementation of the information platform and its associated management functions. The target audience for the design documents produced by this activity are operations staff such as DBAs and Systems Administrators. The detailed design covers the following:

  • Security
  • Archiving data
  • Restoring data
  • Monitoring data usage

Although this activity occurs in Phase 4 of the MIKE2.0 Methodology and is therefore one of the continuous implementation phases, the fundamental solution design of this area oftentimes does not undergo significant change throughout the life of the programme. Therefore, the first increment of the project is typically where the majority of work occurs.

This design will oftentimes reference existing solutions that are already in place across the technology environment.

Major Deliverables
  • Physical Database design aspects not already covered in physical modelling design
  • Disaster Recovery and Failover Design
  • Disaster Recovery Design
  • Access Control, Auditing and Security Design
  • Operations and Monitoring Design
Tasks

Task: Finalise Physical Database Design

Objective: Whereas the Physical Database Design covers aspects of the database design that are primarily related to modelling, there are other design aspects that may need to be give to the Database Administrators. Some common steps for which a process should be designed include:

Table Maintenance

The DBA must manage the creation and archiving of the tables in the information system. Periodically, the DBA will need to create the next period’s partition tables and archive the oldest set of tables (assuming that data partitioning strategies are being used). Using a separate set of tables for each time period simplifies the process by reducing the number of tables involved and allowing the system to "age out" the oldest data.

Database Expansions and Rebuilds

Database Expansion and Rebuilds deal with standard space management issues within databases such as:

  • Space allocation - data file and tablespace allocation
  • Data file placement and configuration
  • Import and exports for performance reasons, i.e., defragmentation
  • Index rebuilds

Much of this information is technical metadata that should be updated into the physical data model as part of a metadata-driven approach.


Input:

  • Non-Functional Requirements
  • Physical Database Model


Output:

Task: Design Backup and Recovery Procedures

Objective: This task produces a deliverable that describes how and when backups of the system will be performed and how those backups will be recovered in case of system failure. Some common techniques that may be used as part of a Backup and Recovery Design include:

Read-only tablespaces

In the case of a warehouse, the use of read-only tablespaces reduces the amount of work involved in a backup process. Once a tablespace has been marked as read-only and backed up, it will be excluded from subsequent backups; most of the warehouse data will be read-only following loading. Similarly, recovery is simplified by allowing the recovery of only the affected tablespaces. These tables can use standard backup and recovery techniques.

RAID Technology

RAID technology may be used to ensure continuous availability of the environment through disk mirroring. It is sometimes more cost effective not to mirror the large volumes of low level transaction data in an information platform, since most of the queries should access the provided summary tables.

High Performance Drives The use of high speed and capacity drives to move data to and from secondary storage is required to minimize the backup/recovery time. The Backup and Recovery design will generally be built on a combination of techniques (mirroring, RAID, etc.) to ensure the highest possible availability and recoverability of the system during the user access and operational windows.

As part of this design, the following information will typically be included:

  • Statement of the critically of the system
  • Scheduled outages
  • Unscheduled outages
  • Risks
  • Technical design of backup process
  • Scripts to be run

Uptime requirements of the environment are a critical factor in determining the proper technology to be used; information platforms generally have more flexible requirements with regards to uptime than operational systems.


Input:


Output:

Task: Design Data Archival and Restoration Procedures

Objective: As data ages, it needs to be purged to clear room for more current data. As the data is likely to be partitioned by time, the oldest partition can be archived, and the data partition reused. Purged data may also need archiving, so it can be accessed if required at some future date.


Input:


Output:

  • Archiving Design

Task: Design Disaster Recovery and Failure Procedures

Objective: The Disaster Recovery Design deals with the technical solution and processes required to handle a catastrophic event that impacts the information environment. The cost/benefit of the system is a key input to this process, as it is quite possible that an alternative system may not be used as part of the Disaster Recovery Plan. In many cases the disaster recovery solution for an information platform involves rolling back to a test environment; advances in the redundancy built into both hardware and software has helped lessen some of the risk factors in this area although true catastrophic risks require sophisticated solutions and often significant cost.


Input:


Output:

Task: Design Access Control, Auditing and Security Procedures

Objective: Access control and security is of critical importance to protecting corporate assets. In addition, an information system must ensure that analysis is based on a complete set of data. Data access controls are needed to protect:

  • Loading the raw data
  • Updating the core data
  • Accessing the core data
  • Accessing other users’ summaries or extracts
  • Controlling the proliferation of data

Controls can be implemented using a combination of the following strategies:

  • Roles
  • Profiles
  • Horizontal and vertical partitioned views
  • Audit tables and database triggers
  • Public and private synonyms
  • Summary extracts sharing

In general, these controls are done mostly within the database itself, with additional controls in the integration and presentation layer.


Input:


Output:

Task: Design Operations and Monitoring Procedures

Objective: Constant monitoring of the information environment and its usage helps ensure its accuracy and usefulness. Typical aspects that may be monitored include:

  • Runtimes for data loads
  • Data growth
  • Types of queries being submitted by the analysts
  • Runtimes for these queries

With this information, the operations team has the information available to effectively manage and enhance the information environment to support the changing needs of the user community. This information can be used for:

  • Spotting trends in data volumes and load times** Better estimating the time required for future loads
  • Identifying additional summary tables
  • Spotting changes in user response times
  • Investigating anomalies

Most products have their own monitoring suites. An important part of the design may involve the integration of these suites into a holistic console.


Input:


Output:

Role:Infrastructure Architect

Role:Infrastructure Developer

(Systems Administrators, DBAs)

Yellow Flags

  • Operations team does not have the capability to effectively support the solution that has been designed – this work must fall on the development team
  • Infrastructure Design does not effectively business requirements, especially those related to security, reliability and recovery
  • Performance and scalability requirements are still operating largely off unknowns and vague estimates

Potential Changes to this Activity

Wiki Contributors
Collapse Expand Close

View more contributors