RIP Roger Kaufman

I’m very saddened by the news that Roger Kaufman, Ph.D. professor emeritus from Florida State University passed away this afternoon. He was a friend & mentor since the early 80s. He gave so much to ISPI & Performance Improvement.

He will be missed. Mega. RIP.

My condolences to his wife Jan.

Roger Kaufman
Roger Kaufman

From Wikipedia:

Roger Kaufman is an American figure in the history of educational technology and performance improvement fields as well as strategic thinking and planning for public and private-sector organizations. Regarded as one of the founding figures of the field,[1] he is referred to as the father of needs assessment.[2]

Kaufman developed the Mega Planning model, a framework for adding measurable value to society.[3] In 2014, the International Society for Performance Improvement created the Roger Kaufman Award for Societal Impact.

Needs Assessment 

One of the defining characteristics of Kaufman’s work is his emphasis on “need” as a noun, not a verb; it is a gap in results and consequences, not a gap in resources or methods. Kaufman explains when “need” is used as a verb, it presupposes a solution before identifying the actual problem to be solved. When using ‘need’ as a verb, an intervention has been selected prior to a clear definition of the actual gap in results that would be addressed. Once a gap, or need, is accurately identified, only then can a means be sensibly selected for moving from current to desired results.

Kaufman expanded this approach to “need” from looking at gaps in products to gaps in outputs and then outcomes: from building block results to results delivered outside the organization to external client and societal results – what the organization used, does, produces, and delivers and the consequences all of that adds measurable value for our shared society. Used in this way, cost-consequences estimates may be made to prioritize closing gaps on the basis of the cost to meet the need as compared to the costs to ignore the need.

Kaufman identified three types, or levels, of needs: Mega, Macro, and Micro.[4] And Change, Choices, and Consequences published by HRD Press. The following table details the levels of needs and their definitions.

These levels of needs are also levels of planning for any organization and indicate a relationship between the levels. Alignment of objectives at each level is critical to ensure that planning translates into clear organizational operations and ensure that activities at each level add value back up the chain linking measurable to societal value-added.[5] As a consequence, no level of results is any more important than the others. Rather, it is the alignment of all levels that is critical to achieving desired results.

Extensive examples of planning and aligned objectives can be found in Moore, 2010 and Moore, Ellsworth & Kaufman, 2008.[6]

Mega Planning 

Kaufman has developed a model for strategic thinking and planning he calls “Mega Planning.” Kaufman argues that many organizational planning models incorrectly begin, and end, with internal or organizational performance and therefore fail to provide organizations a chance to plan how they deliver value outside of their organizations. Traditional planning ends with “Macro” level results, which are organizational results such as profits, graduation rates, ratings, etc. While these are important measures of organizational performance, they do not indicate the impact of an organization on external clients and society. Kaufman’s model is similar to some of the concepts behind “double bottom line” literature.

“Mega Planning” starts with the question of “What kind of world do you want for your children and grandchildren?” with responses distilled in terms of consequences.[7] An Ideal Vision defines the measurable variables for Mega planning including survival, self-sufficiency, and quality of life. He calls Mega planning (a system approach) “Mother’s Rule” because if you ask just about any mother what kind of world they want for their children, they don’t talk to means (credentials of teachers, money spent on social programs) but the survival, health, and happiness of their children.

Basic Ideal Vision for Society – indicators for societal well-being


The world we want to help create for tomorrow’s child:

There will be no losses of life nor elimination or reduction of levels of well-being, survival, self-sufficiency, and quality of life from any source, including (but not limited to):

  • War and/or riot and/or terrorism
  • Unintended human-caused changes to the environment including permanent destruction of the environment and/or rendering it nonrenewable
  • Murder, rape, or crimes of violence, robbery, or destruction to property
  • Substance abuse
  • Disease
  • Pollution
  • Starvation and/or malnutrition
  • Destructive behavior, including child, partner, spouse, self, elder, and others
  • Discrimination based on irrelevant variable including color, race, age, creed, gender, religion, wealth, national origin, or location

Poverty will not exist, and every woman and man will earn at least as much as it costs them to live unless they are progressing toward being self-sufficient and self-reliant. No adult will be under the care, custody, or control of another person, agency, or substance. All adult citizens will be self-sufficient and self-reliant as minimally indicated by their consumption being equal to or less than their production. This becomes the basis for defining a vision and mission statement that map out exactly how an organization, of any type, will add measurable value to society.

Kaufman’s models is argued to be the first model for business planning that makes the business case for social responsibility and that establishes a data-based construct for organizational planning and evaluation that goes beyond the walls of the organization.[8] Recent work by McKinsey & Co’s Ian Davis (the Economist, 2005) aligns with this concept. The model was developed over many years, and cross multiple countries and cultures, to validate the measures. Moore conducted a factor and reliability analysis of the construct, finding that all measures loaded onto a single factor (i.e. described a single concept), r=.95 (a notably high reliability).[9] Kaufman’s model has sometimes been referred to as “Kirkpatrick Plus” – an extension of Kirkpatrick‘s Four Levels of Evaluation[10] by adding Mega—societal value added as a fifth level. However, the debate over this is a rather long-standing one. Some practitioners in the field argue that the fifth level of evaluation is ROI, or “return on investment,” evaluation.[11] (ROI is still a measure of organizational performance internally, however, and does not measure what the impact of an organization is on society).

Examples / Case Studies of Applied Mega

At the Sonora Institute of Technology (ITSON) in Sonora, Mexico, Kaufman and colleagues have worked with the President of the Institute to create a certification, Master’s degree, and Ph.D. based on needs assessment and Mega planning. Students at the Ph.D. level must be working in a company or head of a company and must apply the Mega principles in a real-world project. Evidence of ITSON’s results were first described by Guerra and Rodriguez.[12] ITSON also set up a Performance Improvement Institute for applied R&D where there are currently 20 organizations applying Mega thinking and planning.

Another case study is a project for the president and minister of tourism of Panama taking a systemic approach using the Mega planning model to address issues of high crime, low standard of living, and other measures of social impact. Using the metaphor of “city doctors,” Bernardez, Vallarino, Krivatsy, and Kaufman (2012)[13] identified the “symptoms” of Colon City, Panama, developed a shared vision for Colon based on Kaufman’s Basic Ideal Vision, prioritized, and set goals for the major strategic indicators (such as security, job creation and employment, recovery of real estate values, increase in tourism revenue, and health and sanitation). The team then developed a systemic framework and visualized Colon City as a social ecosystem in order to align actions and make sure actions in each area were coordinated and sequenced with others to avoid conflicts and unwanted consequences (e.g. relocating squatters to places where they could not find jobs). Over a four-year period, at the Mega level, the effort demonstrated increased redevelopment, increased tourism, increased jobs (1,390 jobs in Year 1 and 12,900 new jobs over the 4-year period, 70% of which were permanent jobs thereby reducing unemployment), along with other positive impacts. At the Macro level, Colon City increased revenue from year to year across tourism, real estate, and waste management sectors.

Roger – on Matt Richter and Will Thalheimer’s Truth in Learning Podcast

That starts at about the 44:25 minute mark.

4 HPT Videos

I was lucky to have recorded these 4 video with Roger.





RIP Roger. Thank you for all of your contributions to Improving Performance!

You will be missed.

# # #

Stakeholders Beyond the Customers

Will Thalheimer’s Inaugural Post on HPT Treasures from Friday – here – had me thinking – and then Fred Nichols made a comment about Stakeholders – and I thought I’d re-post here something from my Blog at from 2015 – based on my article, published in the Journal for Quality and Participation in March 1995 – address “Balancing Conflicting Stakeholder Requirements,” that suggests that The Customer is Not the King of Stakeholders (despite the unfortunate slogans from the Quality movement despite Deming’s admonitions about slogans). A PDF of that 1995 article may be found at the end of this post – along with links to other related posts.

The Customer Is King (Not)  

It’s been decades since I saw a poster on the wall at a client site reading exactly or intending to convey that “The Customer is King.”

But I have heard this sentiment spoken or inferred in things I’ve read recently.

A quick test: Are we willing to meet the customer’s requirements at any cost? Even if that requires us to break the law?

The answer is “No!” – and it is no because nothing in business (or anywhere else) is really that simple. The truth is that there are a very complex set of Stakeholders beyond the Customers for any Supplier – be they internal and/or external Suppliers. And another truth is that often the requirements of the various stakeholders are in conflict with each other.

What’s an empowered team of people to do as they confront these realities? Deciding who wins and who loses is a game that few like to play.

Who Are These Stakeholders?

My starter model of Stakeholders will require adaptation for each use. Note that there are many potentially different sub-categories for each and many specific types of organizations and/or people in each of them. The categories are, in alphabetical order:

  • Board of Directors
  • Community
  • Customers
  • Employees
  • Executive Management
  • Government
  • Management
  • Shareholders/Owners
  • Society
  • Suppliers

Would you agree that each of these Stakeholder categories are stakeholders and that they each have requirements (needs) and wishes (wants)? And that those needs and wants could be in conflict?

How do we then balance the conflicting requirements and determine where tradeoffs can and should be made? How do we evaluate them to determine how to create a win-win solution for everyone? How do we conclude whether a win-win for everyone is actually feasible and not just a “pie in the sky” aspiration caught in the real world of variability?

The first thing to do – in my view – is to view them as a hierarchy.

Here is one example in graphic form. Note, your situation might vary – and you will need to adapt rather than adopt this hierarchy.

Stakeholder Hierachy Example 3

What Do These Stakeholders Require?


Borrowing from the work of Roger Kaufman…as presented in Wikipedia…

There will be no losses of life nor elimination or reduction of levels of well-being, survival, self-sufficiency, and quality of life from any source, including:

  • War and/or riot and/or terrorism
  • Unintended human-caused changes to the environment including permanent destruction of the environment and/or rendering it nonrenewable
  • Murder, rape, or crimes of violence, robbery, or destruction to property
  • Substance abuse
  • Disease
  • Pollution
  • Starvation and/or malnutrition
  • Destructive behavior, including child, partner, spouse, self, elder, and others
  • Discrimination based on irrelevant variable including color, race, age, creed, gender, religion, wealth, national origin, or location


They require compliance with laws and regulations. For more specific insights, talk to your executives, the law department, standards groups, and your labor relations organization.


They typically require growth in the value of their shareholder equity. For more insight, talk to your board of directors, public relations, and your finance organization.

Boards of Directors

They require progress toward long-term business goals and strategies and short-term business goals and strategies. For more insight, talk to your CEO and functional/unit executives.

Executive Management

They require much the same as the Board of Directors.

Stakeholder Hierachy Example 2


They require much the same as the Board of Directors.  For more insight, talk to your CEO and functional/unit executives.


They require products or services meeting their requirements at a cost/value ratio that beats your competition. For more insight, conduct a customer survey and talk to your marketing/sales/service personnel who work closely with customers.


They require job security, competitive wages and benefits, a safe work environment, and opportunities for personal challenge and growth. For more insight, talk with your labor relations personnel and all levels of employees. Survey attitudes and concerns by holding formal and informal discussions with your employees or conducting employee attitude surveys. Use suggestion programs.


They require continuity of business, an accurately forecasted demand for their products/services, profitability. For more insight, talk to your suppliers and your materials/purchasing organization personnel.


See the Society requirements, but note that there may be specific local community requirements as well.

And Other

Note: you might also have other Stakeholders, such as standards bodies and unions.

Stakeholder Hierachy Example 1

Recognizing and Balancing Conflicting Stakeholder Requirements

Given the extent of your potential stakeholders and their requirements there are really three challenges.

  1. Identifying specifically who the primary stakeholders are for any given situation.
  1. Understanding their specific requirements and priorities.
  1. Balancing those requirements that are in conflict.

The multitude of requirements can be simplified and communicated visually, using a matrix format showing the key requirements of each of the primary categories of stakeholder.

An Example Stakeholder Requirements Matrix

To construct your own matrix use a spreadsheet program which will make it easier to add/delete columns. See this next example.

Stakeholder Requirements Matrix Example 2002

To complete the matrix:

  1. Identify the stakeholders and their requirements.
  2. The numbers along the horizontal line correspond to those listed vertically. Add additional columns as required to correspond to the number of requirements listed vertically.
  3. Compare each requirement along the left edge to the numbered requirement along the top. Rate those requirements with a high level of conflict at 3, those with a medium level of conflict at 2, and with low conflict at 1. Requirements that do not conflict can be left blank.

Stakeholder Summary

We must understand all of the stakeholders involved and put customer requirements in the proper perspective. And we need to get this message across to our employees.

Good luck. And may the balance of requirements be in your favor.

Note: This post was adapted from my 1995 article:

The Customer Is King – Not! – 15 page PDF – the original version of the article published in the Journal for Quality and Participation in March 1995 – address Balancing Conflicting Stakeholder Requirements, and suggests that the Customer is Not the King of Stakeholders (despite the unfortunate slogans from the Quality movement despite Deming’s admonitions about slogans). Balancing Conflicting Stakeholder Requirements – Wallace – March 1995 AQP

And the version in ISPI’s November 2011 Performance Express: Beyond the Customers The Customer Is King Not

*** *** ***

T&D: Just as Important as Analysis – Alignment with the Stakeholders

About using Project Steering Teams in projects – a video 15:33 minutes in length.

T&D: May the Stakeholders Be With You – By Design

About Design Thinking.

L&D: Is Your Partnership with Your Customers and Stakeholders Formal Enough?

About a Governance & Advisory System and Processes.

# # #

How to Look Good on Virtual Calls

Image by Jagrit Parajuli

At the risk of emphasizing surface features way too much, you’ll find links in this post to two short videos, a Twitter feed, and a blog post. Why share these? To help you feel great when you’re on Zoom (etc.) calls.

Here’s why: because, unless you have mastered this already, you will look better (plus, feel more confident & hence, have better meetings unhindered by wondering if you couldn’t be doing something better with your lighting, etc).

Why did I choose this topic? Because a) I have trouble lighting my space when I’m online and b) because some of my colleagues are accidentally not showing themselves in their own best light (or camera angles), which can be distracting.

So, if you want to fix this issue for yourself, here’s what you can learn from Gia Goodrich (photographer, educator) and Angie (anti-aging, beauty, style, & health video blogger). Some of their topics:

  • Light that flatters (rather than flattens, or worse)
  • Camera angles that make you look your best (possibly with fewer chins)
  • Eliminating distractions for a better meeting (background noise, clutter)
  • Where to look (not staring at the box with the person’s picture in it)
  • What to wear (at least on top)

For more fun with backgrounds, check out Room Rater on Twitter. After the pandemic necessitated that so many people Zoom or otherwise call in from home, Claude Taylor and Jessie Bahrey started rating the backgrounds of various famous people. They started out doing this for fun, and now they have criteria (from 1/10 to 10/10). You can find their criteria and tips that they shared just before judging rooms for the Emmys this year.

HPT Video Weekend Matinee: Lane 2009

The HPT Video Weekend Matinee series is intended to introduce you to the library, with over 100 videos, in the hopes that you’ll share them further into your professional networks, as you see appropriate. And if you have videos to share with us, please forward them to the site administrators.

# # #

Right Wing Ideology at the Heart of L&D?

In this, my first blog post for Guy Wallace’s HPT Treasures, I thought I should start small, really small, atomic-level small. So how about this? Let me conjecture that L&D (Learning and Development, including Performance Improvement) has been engaged for decades in following a radical right-wing ideology put forth most notably in Milton Friedman’s terribly influential free-market manifesto—published this week 50 years ago.

As the New York Time’s wrote this week:

“It was the essay heard round the world. Milton Friedman’s “The Social Responsibility of Business Is to Increase Its Profits” laid out arguably the most consequential economic idea of the latter half of the 20th century. The essay, published in The New York Times Magazine on Sept. 13, 1970, was a call to arms for free market capitalism that influenced a generation of executives and political leaders, most notably Ronald Reagan and Margaret Thatcher.”

The idea at the core of this atomic bombshell is that everything we do as members of organizations should be devoted to maximizing profits—specifically maximizing shareholder value! Our number one goal is specifically NOT:

  • Providing value to customers.
  • Providing value to suppliers.
  • Providing jobs and careers to employees.
  • Enabling self-respect among our employees.
  • Enabling employees to support their families.
  • Contributing to the wider economic good.
  • Contributing to the wider societal good beyond economics.
  • Being good stewards of the natural world.
  • Providing stability so that democracy can flourish.

What Milton Friedman wrote was that businesses should stick to business and any focus on social responsibility was, in his words, “preaching pure and unadulterated socialism.”

In Friedman’s words again:

“IN a free‐enterprise, private‐property system, a corporate executive is an employee of the owners of the business. He has direct responsibility to his employers. That responsibility is to conduct the business in accordance with their desires, which generally will be to make as much money as possible while conforming to the basic rules of the society, both those embodied in law and those embodied in ethical custom.”

This should sound very familiar to you if you’ve been in L&D for the past few years or few decades. We as learning-and-performance professionals have been targeted with this type of message for years, coming from the highest-reputation citizens in our field. I know I preached a similar message when I was a leadership trainer—that our goal as managers was to increase shareholder value. Haven’t you heard messages like these below? Haven’t you said these things yourself?

  • “The ultimate goal of training is business results.”
  • “The ultimate goal of learning is ROI.”
  • “The ultimate evaluation goal, Level 4, is business results.”
  • “As instructional designers we are entrusted with shareholder equity.”
  • “The goal is not learning, it’s business impact.”
  • “It’s all about increasing shareholder value.”
  • “Training is not an end in itself. Training is a means to an end.”
  • “It’s about aligning learning to impact.”
  • “We should work backward from the ultimate goal, business results.”
  • “Too many learning interventions are created without any consideration of the impact on the organization.”
  • “In designing learning, we should start with a targeted business impact.”
  • “We work for the C-Suite.”

Essentially, what we’ve been doing in L&D over the past three or four decades is advocating for the raving machinations of a right-wing nut job! Friedman’s arguments are now largely taboo, discredited as being too narrow, too unfair, too unbalanced—and wrong! Even the Business Roundtable, a group of American CEOs focused on business interests, have admitted that the shareholder-only-focus of previous years was inappropriate, that the purpose of corporations should NOT just be profits. In their 2019 statement they affirmed “the essential role corporations can play in improving our society when CEOs are truly committed to meeting the needs of all stakeholders.”

As Nobel Prize winner Joseph Stiglitz has written:

“By the time he wrote this essay, Friedman, who had done distinguished analytic and empirical work in economics, had become largely a conservative ideologue. I gave a talk at the University of Chicago around [the time of Friedman’s essay], presenting an early version of my research establishing that in the presence of imperfect risk markets and incomplete information — that is, always — firms pursuing profit maximization did not lead to the maximization of societal welfare. I explained what was wrong with Adam Smith’s invisible-hand conjecture, which said that the pursuit of self-interest would lead, as if by an invisible hand, to the well-being of society. During the seminar, and in extensive conversations afterward, Friedman simply couldn’t or wouldn’t accept the result; but neither, of course, could he refute the analysis — it has been a half-century, and my analysis has stood the test of time. His conclusion, as influential as it was, has not.”

You can read commentary about Friedman’s essay, including Stiglitz’s reflections at

Downfall of Our Professionalism

We have made ourselves subservient to our business masters! Yes, read that sentence again!

More importantly, think of the implications. When we are just cogs in the larger machine, we don’t exert our own direction; we circle round and round and round based on the whims and self-serving interests of our overlords. We are not professionals then, are we? We are robots, following the forces of algorithms, business incantations, managerial incentives.

Professionals have principles, validated practices, unshakable values. As cogs, we hope only for a thin veil of lubrication to ease the grinding servility of our work.

Think about it. Do we stand up for principles or kowtow to our business stakeholders? How many times have you been asked to build learning with no needs assessments or inadequate evaluations? How many times have you used learning designs that you knew were ineffective? How many times have you fallen to your knees as an order taker, when you knew that training wasn’t the real issue? Yes! We know we have a professional responsibility to do what’s right, but we stand down. Almost always! We comfort ourselves in fighting the good fight. We rationalize our place in the machine, thinking that we are lucky to have jobs, thinking that in the rightful order of things our job is to do what we can to increase profits. We are beholden to our bosses, who are beholden to their bosses; all of us are beholden to the business, which is beholden to shareholder value.

We build union-busting training without even thinking that we are hurting our fellow workers. We create onboarding programs that unrealistically show the organization at its best, failing to prepare new hires for the obstacles they will face. We create compliance training we know won’t help our fellow employees stay safe—except for the lawyers whose jobs will endure.

Who are we?

Bright Spots

Certainly, there are acts of bravery and professionalism in the field. One of my heroes is Roger Kaufman, who has preached the importance of looking beyond the organization to other goals of learning and performance improvement. His work on MEGA is exemplary (

In my LTEM Framework (, I specifically point out that organizational outcomes are only one outcome to be measured. We ought to also look at our learning interventions’ impact on other stakeholders—including the learners themselves; their coworkers, family, and friends; the community; society; and the environs.

But to date, these efforts are too small and insignificant. We need more structures and practices and values focused beyond business results and shareholder value. We also need to empower ourselves to be professionalized, so we can stand up and do what is right.  

Did You Know You Were Spouting Right-Wing Ideology?

You and I should worry. It’s easy to see things as they are as immutable and righteous. I bought into the dogma of shareholder value. I have focused too stridently on business results. I have been a helpful servant. I am not proud of this.

I write this post because I see a connection between politics-and-power and the work that we do—and the justice and fairness and empowerment that may or may not arise from our work. We may want our work to be siloed away from concerns about power and politics, but just because we haven’t noticed a connection, doesn’t mean there isn’t one.  

It sure looks to me that the messaging we’ve heard and repeated to ourselves over the years is a message that pushes us away from thinking about our fellow employees, that focuses us on financial outcomes for a narrow few, that binds us to a state of servility, that makes us less than the professionals we yearn to be.


This is Will Thalheimer, President of Work-Learning Research, Inc.
You can connect with me here:

Fifteen Common but Questionable Principles of Multimedia Learning

By Richard E. Clark, David F. Feldon, and Soojeong Jeong


The chapter begins with a quick summary and research update on earlier lists of ten
questionable multimedia principles (Clark & Feldon, 2005, 2014). We then add five
additional principles that have gained traction in recent years.

The updated 10 former questionable beliefs include the unfulfilled expectations that multimedia instruction: (1) yields more learning than live instruction or older media, (2) is more motivating than
other instructional media, (3) provides animated pedagogical agents that aid learning, (4)
accommodates different learning styles and so maximizes learning for more students; and
also benefits learning by allowing and encouraging (5) student managed constructivist
and discovery approaches, (6) autonomy and control over the sequencing of instruction;
(7) higher order thinking skills (8) incidental learning of enriching information (9)
interactivity, and (10) authentic learning environments and activities.

The more recent additions and the focus of this discussion are false expectations that multimedia
instruction benefits learning by providing: (11) presence from virtual reality sessions,
(12) virtual and remote laboratories, (13) gamification, (14) recorded lectures, and (15)
brain training.

We end the chapter with a suggestion for using big data and aptitude treatment interaction research to avoid or resolve many past and future mistaken principles.

Free 30 page PDF here: Clark Feldon Jeong 15 Mistaken Principles Mayer 3rd ed 2020 final

# # #

Article: Type Hype? Why many researchers are thinking twice about the Myers-Briggs Type Indicator


Let’s face it: A lot of entrepreneurs, small businesses, and even some large ones make a lot of money selling some form of personality stereotyping instrument. If you become certified to administer the assessment and pay to be able to facilitate workshops around them, companies, especially the large ones, will pay a lot of money for your services.

Some of these typecasting tools are fads. Others are “home-grown.” A few, though dominate. In Type Hype, Richard E. Clark focuses on the most widely used inventory, the Myers-Briggs Type Indicator (MBTI). In the article, Clark shares “four reasons researchers and the National Research Council of the National Academy of Sciences question the test’s validity.”

The article is two pages and is an excellent summary of the four key arguments.


You can find the article in HPT Treasures: On the Clark page, there are 40 additional articles: