Knowledge management assessment tool

Definition

A tool to help identify strenthsand development areas in the organization's overall KM framework. This by IAEA recommended tool is often used for knowledge management maturity assessment.

Description

Individual criteria have been identified that are considered as key elements towards an effective approach to KM. These criteria have been grouped into seven organizational or functional categories, to facilitate the self-assessment, via:

1. Policy/Strategy
2. Human Resource (HR) Planning and HR Processes
3. Training and Human Performance Improvement
4. Methods, Procedures & Documentation Processes for Improving KM
5. Technical (IT) Solutions
6. Approaches to Capture/Use Tacit Knowledge
7. KM culture/Workforce Culture Supporting KM

Metrics for the overall self-assessment and for each individual category have been developed as shown below:

KM Self-Assessment Metrics Scoring

Rating	Extent Currently	Extent Should Be
0	Not utilized at all	Not utilized at all
1	To a little extent	To a little extent
2	To some extent	To some extent
3	To a great extent	To a great extent
4	To a very great extent	To a very great extent

A self assessment questionnaire has been developed based on the above seven categories. A Microsoft Excel based tool is also available that is used to facilitate the self assessment process. The tool uses radar/spider diagrams for each of the seven functional categories and at an executive summary level to give management a graphical depiction of current KM strengths and future development areas. An example of the output is given below:

Example output

The assessment tool can be used independently by a nuclear operating organization for an internal review, as a prerequisite for a KM assist mission or during a KM assist mission. These criteria are not so much intended to provide a “report card” as they are to assist managers in identifying strengths to build upon and areas for improvement to be addressed in the knowledge management area.

Policy/Strategy

This topic covers the following aspects:

• Written policies/strategies
• Communication strategy
• Identification of KM responsibilities

For background information see References [1–7].

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Does the organization have a written policy for implementing its strategy in KM area?
2	Is a KM policy integrated into the management system?
3	Is the KM policy communicated to all staff in the organization?
4	Are those responsible for managing the formulation and implementation of the organization KM strategy clearly identified?
5	Does the organization’s strategic focus support continuous learning to improve individual and organizational performance?
6	Is the organization’s KM policy aligned with continued emphasis on a strong safety culture?

An example of the output (in form of a radar/spider diagram) for the category 1 is given below

Example output

Human resource (HR) planning & HR processes

This topic covers the following aspects:

• Workforce planning
• Succession planning
• Risk assessment for critical knowledge loss
• Employee development plans for KM

For background information see References [3, 4, 6, 8–12]

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Does the organization have a written policy for implementing its strategy in KM area?
2	Is a KM policy integrated into the management system?
3	Is the KM policy communicated to all staff in the organization?
4	Are those responsible for managing the formulation and implementation of the organization KM strategy clearly identified?
5	Does the organization’s strategic focus support continuous learning to improve individual and organizational performance?
6	Is the organization’s KM policy aligned with continued emphasis on a strong safety culture?

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Does the organization implement a comprehensive methodology to ensure that HR needs both current and future are met (work force planning)?
2	Is there an effective succession planning programme in place?
3	Are risk assessments carried out to identify potential loss of critical knowledge and skills?
4	Are exit interviews carried out to capture critical knowledge and experience when people leave the organization?
5	Does a programme exist to develop new leadership /technical talent in a timely manner?
6	Does the organization utilise job profiles or equivalent to assess and monitor its skills/competency needs?

An example of the output for the category 2 is given below:

Example output

Training and human performance improvement

This topic covers the following aspects:

• Coaching and mentoring
• SAT
• Simulator use
• CBT (e-learning)
• Refresher training
• Human Performance Improvement

For background information see References [4, 6, 9, 10, 13–19]

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Does the organization incorporate formal Systematic Approach to Training (SAT) principles into its training programmes?
2	Does the formal SAT programme address capture and dissemination of knowledge?
3	Does the training programme utilise appropriate tools such as simulators, Computer Based Training (CBT), multi-media simulations, etc. to capture/transfer critical knowledge?
4	Is competence evaluated on a regular basis?
5	Is regular refresher training carried out to maintain and enhance competence?
6	Does the organization have a formal human performance improvement programme to maintain and enhance competence?
7	Are coaching and mentoring approaches used to support knowledge sharing?

An example of the output for the category 3 is given below:

Example output

Methods, procedures & documentation processes for improving KM

This topic covers the following aspects:

• Learning from Operating Experience
• Work control methods
• Error prevention
• Document control/Configuration
• Corrective action programme
• Benchmarking

For background information see References [3, 4, 18–27]

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Are KM methods incorporated into procedures and processes rather than being separate add-on tasks?
2	Does the organization have a comprehensive methodology that addresses learning from experience?
3	Are self assessments regularly used to enhance organizational knowledge?
4	Is external benchmarking regularly used to enhance organizational knowledge by adopting good industry practices?
5	Is the feedback (internal and external) from operational experience (lessons learned) used by the organization for corrective action planning to achieve improvements?
6	Is the composition of work teams (such as individual expertise/experience) considered in order to enhance knowledge transfer?
7	Are all work activities documented in such a way that knowledge can be effectively retrieved, shared and utilized?
8	Are procedures, drawings, lesson plans and related documentation updated promptly in a systematic way to address technical and organizational changes?

An example of the output for the category 4 is given below:

Example output

This topic covers the following aspects:

• Knowledge data bases
• Content/document management systems
• Search engines
• Portals/Intranet
• Wikis/blogs
• Skill/competency databases
• Expert yellow pages
• Enterprise Resource Planning (EPR)
• Other IT supporting systems

For background information see References [3–5, 21, 28, 29]

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Are IT and KM strategies aligned?
2	Is the organization utilising an integrated approach in managing its information?
3	Does the organization utilise appropriate IT support systems and tools such as:
3.1	Content/document management
3.2	Concept mapping
3.3	Knowledge databases
3.4	Simulation tools
3.5	Enterprise Resource Planning (ERP)
3.6	Portals/Intranets
3.7	Knowledge search engines
3.8	Expert yellow pages
3.9	Expert systems
3.10	Wiki’s/blogs

An example of the output for the category 5 is given below:

Example output

Approaches to capture/use tacit knowledge

This topic covers the following aspects:

• Taxonomy development • Process for critical knowledge ID • Processes for knowledge elicitation/harvesting • Concept mapping • COPs • Coaching & mentoring

For background information see References [1, 4, 5, 8].

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Does the organization utilise methods to identify people who have critical knowledge?
2	Does the organization adopt effective techniques to capture critical knowledge such as:
2.1	Elicitation interviews
2.2	Video capture
2.3	On the Job Training (OJT) dialogue
2.4	Mentoring/coaching
2.5	Communities of Practice (COP)
2.6	Explicit capture (narrative documentation)
2.7	Card sorting (manual concept map)
2.8	Concept mapping
2.9	Process mapping
2.10	Story telling
2.11	Others
3	Is information and data retained and presented in an effective way to facilitate search and retrieval?
4	Does the organization have processes for the effective transfer and utilisation of captured knowledge?

An example of the output for the category 6 is given below:

Example output

KM culture/workforce culture supporting KM

This topic covers the following aspects:

• No blame environment
• Sharing knowledge
• Leadership/commitment

For background information see References [1, 3, 4, 19].

No.	Assessment Criteria/Questions	Extent currently utilized									Extent should be utilized	Comments
		0	1	2	3	4	0	1	2	3	4
1	Does the culture of the organization promote the sharing and transfer of knowledge, particularly tacit knowledge, amongst personnel?
2	Does the organization have an open, no blame approach to reporting incidents/events and sharing from lessons learned?
3	Is sharing of knowledge in the organization recognised and rewarded?
4	Do managers lead by example performing practical, visible leadership supporting the knowledge management strategy?
5	Do managers encourage trust, cooperation and collaboration between individuals and teams?

An example of the output for the category 7 is given below:

Example output

References

• [1] INTERNATIONAL ATOMIC ENERGY AGENCY, Planning and Execution of Knowledge Management Assist Missions For Nuclear Organizations, IAEA- TECDOC-1586, IAEA, Vienna (2008)