Agile Courseware - How Teams Can Make More Learning Experiences in Less Time


*Agile Courseware *

How Teams Can Make More Learning Experiences in Less Time




Raytheon Intelligence, Information & Services (IIS) Engineering and Technology, Training & Logistics Services


Table of Contents

1. Preface

1.1. Copyright

1.2. Acknowledgements

1.3. About the Author

2. Conventions Used

3. Introduction

4. Why Care

4.1. Intended Audience

4.2. Why a Training-Specific Book about Lean-Agile

4.3. A Brief Note to Chief Learning Officers (CLOs), VPs of HR or VPs of Engineering

4.4. Why Care about Lean-Agile as Buyers of Training Products and Services

4.5. Why Care about Lean-Agile as L&D Professionals

5. Setting the Stage

5.1. Readers Familiar with Lean-Agile in Software can Skip this Chapter

5.2. A Brief History of Applying Ideas from Other Domains

5.3. A Brief History of Agile

5.4. A Brief History of Lean

5.5. A Brief History of the Theory of Constraints

5.6. A Brief History of the PDCA Cycle

5.7. Visualizing Workflow

5.8. Trello as a Virtual Kanban Tool

5.9. Lean-Agile is a Big Skill that Combines Middle Skills

5.10. Your Organization May Vary

6. Proprietary Versus Open

7. Focusing on Buyers of Training

7.1. Traditional Sequentially Executed ADDIE (aka Waterfall Method)

7.2. A Brief Comparison Between the Lean-Agile approach and ADDIE

7.3. Determining How Much Lean-Agile a Buyer Organization can Apply

7.4. How the Lean-Agile Approach Impacts the Buyer’s Risk

7.5. How Agile Changes the Buyer’s Training Procurement Process

7.6. What the Buyer Should Tell their Staff to Expect from Lean-Agile

7.7. How Training Buyers Can Determine Who Really Knows Lean-Agile

7.8. How Much More Time will Lean-Agile Take for Buyer’s Staff

7.9. Behold Complete Transparency

7.10. Lean-Agile and Earned Value Compatibility

8. Adjusting to an Agile Mindset

8.1. Don’t Wait for Perfection

8.2. Leading a Lean-Agile Implementation

8.3. Workplace Training Rather than Education

8.4. Lean-Agile Modifies ADDIE Rather than Replaces ADDIE

8.5. Lean-Agile is Not a Silver Bullet

8.6. The Cottage Industry vs Mass Production Model

8.7. Adopting Lean-Agile – Quickly or Slowly

8.8. A Lean-Agile Journey

8.9. Driving Assumptions, Context and Constraints

8.10. Lean-Agile is Still New to Many Training Product Buyers

8.11. Complexity Impacts Training Projects Today

9. Principles of Agile – The Key to Success

9.1. Principles Versus Methods

9.2. Agile Principles

9.3. Lean Principles

9.4. Theory of Constraints Principles

9.5. Minimizing Feedback Delay

9.6. Make Work Visible

9.7. Geographically Dispersed Teams Versus Colocated Teams

9.8. The Seen and the Unseen – Noticing Lean Waste

9.9. Working Courseware Increment

9.10. Respect for People

9.11. Small Batch Training at the LO Level

9.12. Standard Work in the Form of Checklists

9.13. Make Defect Demand Visible

9.14. Customer Collaboration for Earlier Feedback

9.15. Continuous Improvement – Kaizen

9.16. Planning Still Happens in Lean-Agile

10. Adaptations to Agile for Training Development

10.1. Lean-Agile Design–Still Experimenting–Stay Tuned …

10.2. Kanban is the Most Flexible Workflow Tool Yet

10.3. Kanban as a Process Map

10.4. What to Track on your Kanban Board

10.5. Requirements as Work Items

10.6. Build the Courseware Backlog

10.7. The Courseware Initial Plan

10.8. Don’t Start with Velocity Tracking

10.9. Specialization Expectations for Training Development

10.10. Why We Call it a Courseware Increment Instead of a Lesson

10.11. Lean-Agile Works for Related Product Development

10.12. Degree of Formality – CDD or a simpler List of LOs and Strategies

10.13. Translating Training Agile Terms for Software Developers on your Team

10.14. Content Versus Players

10.15. Communicating Kanban Defect Cards to the Customer

10.16. Simulation Development Uses Agile Like Software Development

11. Adaptations for Customers that Reject Agile

11.1. Using Kanban In-House Even if a Customer Requires Waterfall

12. Troubleshooting Agile Problems

12.1. Diagnosis Questions

12.2. Experimenting with PDCA

12.3. Until Team Members Get Pull – Overcoming Push

12.4. Fixing Bottlenecks

12.5. Dealing with Customer Demand Fluctuations—Determining Labor Capacity

12.6. Accounting for Variances from Plans

12.7. Some Customers are Not Used to Fixed Time Box Iterations

12.8. Variance is Not the Enemy in Product Development

12.9. Using Queuing Theory to Improve Kanban Traffic Jams

12.10. Capture Troubleshooting Lessons Learned for Other Teams

12.11. Troubleshooting in Obvious, Complicated, Complex and Chaotic Domains

12.12. Other Troubleshooting

13. Mistakes and Misconceptions Others Made So You Don’t Have To

13.1. Team Members Working on Too Many Things at Once

13.2. Ensure your Contract Allows Scope Reduction by Prioritization

13.3. When SME Delays Last Beyond One Iteration

13.4. Scaling to Many Teams Means Sharing Lessons Learned

13.5. Good People are Always in Demand

13.6. Some SMEs Don’t Know What to Expect from Lean-Agile

13.7. Kanban May Not Help as Much for Teams of One

13.8. Stakeholder and Organizational Resistance to Lean-Agile

13.9. The Urge to Batch Larger to Avoid Many Kanban Cards

13.10. Messages about WIP Limits and Inventory Waste Don’t Get Through in a Single Telling

13.11. Customers Familiar with Pure Scrum for Software Development May have Misconceptions

13.12. Sometimes Customers Still Want the Courseware Before It Can Be Completed

13.13. Without Learning the Principle, Some Team Members Just Go Through the Motions

13.14. Breakdown Video Production into Smaller Work Items

14. Successes

14.1. How to Convince a Customer to Accept a Lean-Agile Approach

14.2. Early Success Helps Gain Momentum

14.3. Stack the Team

14.4. Productivity from Team Rhythm

14.5. Set Up the Board for the Team, and They Get It Quickly

14.6. Visual Management Sets the Conditions for Kaizen

14.7. Checklists as Job Aids

14.8. Working Product Increments Build Customer Trust in the Team

14.9. Build-In Quality using Checklists

14.10. Scaling to Many Teams Spread Over Many Sites

14.11. Adapting to Organizations that Enforce Linear Waterfall Methods

14.12. Many Heads are Better Than One – Innovation by Involving the Whole Team

14.13. Adapting Lean-Agile to Fit Within CMMI

14.14. Pull Versus Assigning (Push) Works Better

14.15. Skip Kanban Columns When Needed

14.16. Iteration Demos Unify the SMEs

14.17. Training Already has a Way of Decomposing a Work Effort and Estimating

14.18. Find a Virtual Kanban Tool that Allows Batching Card Moves

14.19. Planning Staffing Capacity – Matching Supply and Demand

14.20. Start Agile and Become Lean

14.21. Kanban and Daily Standups Provide Accountability Quickly

14.22. Learning From Each Other

14.23. Share Lessons Learned

14.24. Share Successes with Stakeholders and your Organizational Leadership

14.25. Persuade Organizational Leadership to Underwrite Experimentation Initially

14.26. Track Risk Mitigation and Opportunity Capture Actions as Task Level Cards on the Kanban Board

14.27. Build Lean-Agile into your Process for Getting New Business

14.28. A Few Metrics Help, Too Many Hinder

14.29. Inspect What you Expect

14.30. Reduce Process Complexity Wherever Appropriate

14.31. Standardizing Lean-Agile is Not Antithetical

14.32. Test Process Changes with Experiments Before Going Bigger

14.33. Have Someone Focused on the Customer

14.34. Great Staff is the Root Cause of Success

14.35. Our Dissenter Helped Us Improve

14.36. Welcome Novel Ideas and Solutions from Team Members

14.37. Tiered Kanban Boards Help Show the Big Picture

14.38. Teams Can Change the Process

14.39. Iteration Reviews are “No Homework” Activities for Buyer/Customer Stakeholders

15. Roles in Agile

15.1. Role of the Agile Coach

15.2. Role of the Agile Team Member

15.3. Role of the Agile Quality Assurance Content Inspector/Functionality Tester

15.4. Role of the Product Owner

15.5. Role of the Team Lead

15.6. Role of the Manager

15.7. Role Adaptations for Small Shops

15.8. A Day in the Life

16. Next Steps

16.1. Separate Teams for Player Development and Content Development Teams

16.2. Shape Customer Expectations Before Contract Execution

16.3. Senior Leadership Has Got Your Back

16.4. Getting Started–Make Your Own Way

17. Contact Information

17.1. Contact Information if Assistance is Desired

17.2. Collaborators for this Book

17.3. We Used a Kanban Board to Write this Book

18. Colophon

Appendix A: Lean-Agile Job Tasks for the Buyers

A.1. Job Tasks for Lean-Agile Courseware Buyers

Appendix B: Lean-Agile Job Tasks for the Producers

B.1. Job Tasks for Lean-Agile Development Teams

Appendix C: Markup Technologies

Appendix D: Further Reading

D.1. Further Reading

19. Glossary and Acronym List

19.1. A

19.2. B

19.3. C

19.4. D

19.5. E

19.6. F

19.7. G

19.8. I

19.9. J

19.10. K

19.11. L

19.12. O

19.13. P

19.14. Q

19.15. R

19.16. S

19.17. T

19.18. U

19.19. V

19.20. W

20. Notes


1. Preface

Published by Raytheon Intelligence, Information and Services (IIS), Engineering & Technology, Training & Logistics Services, Dulles, Virginia

ISBN: 978-0-692-66522-0 (ePub version)

Produced in the United States of America

ePub validation successful at http://validator.idpf.org

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Many eBook readers have the ability to open a picture to a larger version by double tapping on the image. We tested that this works for this ePub eBook using the Apple iPad iBook app. Amazon Kindle readers also have this feature. You may convert this ePub to the Kindle format using Calibre (https://calibre-ebook.com) if you prefer the Amazon readers ePub works everywhere else. We only produced the flowable ePub version. See the Colophon for details.|

Non-Export Controlled

This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

[]1.1. Copyright

Copyright, 2016, Raytheon Company. All rights reserved.

[]1.2. Acknowledgements

Thank you to the many practitioners of Agile in the software community, practitioners of Lean in manufacturing from the Toyota Production System and those who later applied Lean to software and other knowledge work for sharing their decades of experience and results on the Internet.

Thank you to other early adopters of Agile in learning and development for your conference presentations explaining what you have done so far.

[]1.3. About the Author

The Raytheon Intelligence, Information and Services (IIS) Engineering and Technology, Training and Logistics Services organization authored this book to share what we have learned as one of the training industry’s first movers in applying Agile to Training Product Development efforts. The Raytheon IIS Engineering and Technology, Training and Logistics Services is made up of hundreds of training professionals.

Our parent company, the Raytheon Company, has 61,000 employees working in four main businesses that netted $23 billion in sales in 2014. Raytheon is headquartered in Waltham, Massachusetts. Our parent business unit is Raytheon Intelligence, Information and Services (IIS), which is a $6 billion (2014) company with 17,000 employees, headquartered in Dulles, Virginia, USA. Some of the things Raytheon IIS does includes the following:

  1. {color:#000;}Providing training to prepare people for the world’s most important missions

  1. {color:#000;}Creating cybersecurity and advanced intelligence solutions that strengthen our global customers’ critical infrastructures, information systems and mission
  1. {color:#000;}Supporting national security objectives by providing comprehensive technical, analytical and operational support to the intelligence community
  1. {color:#000;}Providing full life-cycle mission operations, engineering, sustainment and modernization services across all domains, as well as multi-intelligence ground systems and unmanned systems technology
  1. {color:#000;}Implementing leading-edge security standards for civil and defense space ground programs

The Raytheon IIS Mission states “We partner with our global customers to provide innovative mission support, engineering, intelligence, training and cybersecurity solutions.”

Raytheon designs, develops and delivers training solutions that address customer requirements. Raytheon trains 2 million students per year for industry and government customers worldwide, making Raytheon a world leader in training.

Raytheon uses traditional instructor-led, blended and technology-based training solutions; provides safe and secure training environments; and increases trainee proficiencies for the most complex training programs in the world.

Raytheon supports hundreds of sites worldwide for training, operates or supports over 300,000 training devices at near 100% availability, and conducts over 1.4 million training events worldwide.

Raytheon draws on the experience of thousands of training professionals who deliver over ten million training hours annually and provide targeted education and training solutions. We provide solutions for training for a wide range of industry and government customers. Our extensive experience providing U.S. and international training results in cost-effective training methods that increase functionality across all requirements.


Customer Success is our Mission ™

[]2. Conventions Used

The following conventions are used:

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Notes are used to highlight important information or to present asides relevant to the topic at hand.|
|={color:#000;background:rgba(0, 0, 0, 0);}. Tip|<{color:#000;background:rgba(0, 0, 0, 0);}. Tips provide helpful information on how to most effectively use a particular idea.|
|={color:#000;background:rgba(0, 0, 0, 0);}. Caution|<{color:#000;background:rgba(0, 0, 0, 0);}. Cautions alert you to potential difficulties that may occur.|

[]3. Introduction

Opening this book means you have some curiosity about what Lean and Agile have to do with training. So whether you’re brand new to Agile or have years of experience, we have something for you. If brand new, you may want to review the glossary at the back of the book.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{background:rgba(0, 0, 0, 0);}.
p<{color:#000;}. By convention in this book, we will use “Lean-Agile” as a term to describe our use of Agile and Lean. Some software practitioners, like the scaledagileframework.com, also use the term Lean-Agile.

This book is aimed at learning & development practitioners. Neither Lean nor Agile originated within the learning discipline. We are borrowing successful principles from the domains of software and manufacturing and crossing them over into the learning discipline.

New software engineers take in their discipline’s body of knowledge which already includes Agile. Software practitioners know that Agile practices already include much Lean. New systems engineers take in their discipline’s body of knowledge which already includes Lean.1 New learning and development practitioners take in their discipline’s body of knowledge, which until recently has not included much Lean or Agile. Also, learning and development practitioners may not have much experience in the software domain. So, although some software practitioners may disagree with our usage of the term Lean-Agile because Agile practices already include some Lean, we choose to use Lean-Agile to emphasize that both Lean and Agile are complementary approaches to our learning and development audience that is largely new to both.

This book takes the principles and ideas from Agile Scrum, Kanban and Lean and translates it to the knowledge work of learning experience development.

Our translation of the principles of Agile and Lean into the learning and development domain may not always match one-for-one with the application of these principles in software, manufacturing or systems engineering. Similar to effective communications between two human languages, there may not always be a literal, one-for-one translation.

We expect the learning and development discipline will continue to expand on, and improve methods that build from the principles of Lean and Agile that are tailored to the context of learning and development. |

Buyers, stakeholders and producers of made-to-order training products and services often experience frustration with the length of time it takes to go from the training need or problem identification to an effectively implemented learning intervention.

We have seen Lean-Agile help with shortening that time enough to make your time reading this book worthwhile. Also as more stakeholders want more technology components integrated in learning experience packages, courseware, materials, and blended experiences, Lean-Agile also provides a way of more successfully managing these technology components during development.

Did you know that some organizations have gotten between [* 15% to 50% improvements in their product development cycle time ] and significant *reductions in risk using Lean and Agile? Although these figures are from people applying Lean-Agile in a different domain than learning and development, we were impressed enough by their gains to see if we could get similar improvements for learning experience development. Do you want to hear more about how Lean-Agile can provide you with significantly faster cycle time when buying and making learning experiences as products? We wanted similar productivity improvements, and now you can have these gains too. Do you wonder what Lean-Agile has to do with traditional ADDIE, or analysis, design, development, implementation, and evaluation? We did too. We’ll share some of what we learned and how we use ADDIE with Lean-Agile.

We wrote this book because we think that the learning and development industry can benefit from an additional successful approach that has been used in other industries for 10–25 years. From monitoring the learning and development industry conferences, blogs, events, and happenings, we have noticed a small group of learning and development professionals that have started to apply these Lean-Agile approaches, and we thought we might get some lift from this, too. So, we experimented and we improved, and now we use Lean-Agile in current learning experience development projects/programs for our customers. We have learned a lot about what works and what does not work.

We hope that more people in the industry will (1) consider and explore the use of Lean-Agile in their buying and making of learning experiences as products, and (2) will share what they learn too, so we all gain as a result. We will continue to monitor learning and development conference presentations, articles and blogs to see what our discipline has learned together.

We see Lean-Agile as a collection of various principles and practices, which we later list, where the whole is more than the sum of its parts. We will share what we have found to be the key principles and some of the supporting methods, and techniques of Lean-Agile that you or anyone can apply to develop your successful step-by-step approaches for Lean-Agile learning experience development. We’ll point you towards the giants in the Lean and Agile communities on whose shoulder’s we’ve stood, so you may go directly to the sources for further information.

We share our experiences from our journey and many of our lessons learned that may benefit you as you try to apply Lean-Agile in your own work when buying learning experiences or making learning experience products for others.

Although we cannot share our specific, detailed, step-by-step, “reduction to practice” methods and recipes for Lean-Agile, because it is our intellectual property (IP), we still want to share as much as we can that we think may be of value to other people involved in the making and buying of training products and services. So we will have to be generalized with principles instead.

We think there is value in this book for many types of learning and development related organizations.

If you are a senior level buyer of training/learning experiences, you may not be as interested in all the details that we provide for producers, but hopefully when you delegate this book to your staff, one of them will recognize the value of learning from others. As you progress down a new path, there is an immediate benefit in learning from others when they share some of what they discovered in the process. Then, you don’t have to learn it all from first-hand experience.

We won’t try to make the substantial changes you’ll need to make sound trivially simple. Lean-Agile is not magic, nor does it solve all project/program problems. There is work involved. All good things seem to require effort. We say that the payoff is worthwhile. We will show you some of our efforts and share some of our journey, so you can learn from what others have done and perhaps use that to improve your own journey.

Fundamentally, we see this book as a sort of extended session in a learning and development industry conference like the Association of Talent Development (ATD) Techknowledge, the eLearning Guild’s DevLearn or FocusOn Learning Conference & Expo, for example. We gain much from other conference presenters sharing their experiences, and we hope that our book offers the same benefit for you.

Lastly, we hope to read your works or hear from you at conferences after you have made progress down this path. We are always improving, too, and working to apply any saved time towards better learning outcomes. When the entire community of practitioners shares with each other, we all gain. Sharing, when skillfully applied, can translate to better learning experiences and outcomes in workplace training.

Thank you for taking the time to engage with us in our book. We look forward to your book, article, blog or conference presentation next.

[]4. Why Care[]

[]4.1. Intended Audience

We anticipated two primary audiences:

  1. {color:#000;}Buyers of learning solutions – People who procure learning experience products for their organizations

  1. {color:#000;}Producers of learning solutions – People who produce learning experiences for other people, as a product, and who are often organized into multi-disciplinary teams. The producers audience also includes people who produce learning experiences for their own organizations rather than for others.

[]4.1.1. Buyers of Learning Experiences

By buyers, we mean people who procure learning experiences and training products for their organizations.

This may include the following:

  1. {color:#000;}Government organizations (United States or other countries)

  1. {color:#000;}Commercial for-profit organizations (worldwide)
  1. {color:#000;}Non-profit organizations (worldwide)

[]4.1.2. Producers of Learning Experiences

By producers or makers, we mean learning professionals who produce learning experiences for other people, as a product, and who are organized into multi-disciplinary teams. We also include people who produce learning experiences for their own organizations rather than for others.

The target audience’s development team typically includes one or more instructional designers (there is a nascent trend to calling them learning experience (LX) designers); one or more media people for graphics, animations, video and audio; software developers for HTML5, JavaScript, CSS and sometimes other programming languages for learning content players, game engines and simulations; editors and sometimes technical writers. Also, there are typically subject matter experts, or SMEs, that are assigned to the project.

Sometimes the teams are bigger and may have different specialties. For example with game-engine based learning, we have worked with technical artists that build 3D models, software developers to write the code that provides all the behaviors depending on the complexity, levels and missions. We have also worked with game designers, and now the group can include motion capture specialists.

For complex learning simulations, there are software developers and simulation model designers to make the Sim engine meet the fidelity expectations. Instructional/LX designers tend to lead these cross-functional teams because learning drives the ends, and so they are involved in the means.

If you are a sole producer, you may want to consider combining (a) David Allen’s ideas in his book, Getting Things Done and (b) Kanban. Lean still applies and you can gain from it.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Because Lean-Agile comes with its own lexicon, the glossary at the back of this book explains what we mean by these terms.|

[]4.2. Why a Training-Specific Book about Lean-Agile

Our aim with this book is to help you see that Lean-Agile principles can help buyers and producers in performing and tracking the development of learning experiences. This book also translates the software version of Lean-Agile into what is more appropriate for developing learning experiences. Lean-Agile becomes even more helpful as our learning experiences incorporate more and more complex technology components, our development teams expand beyond traditional roles to include more technology specialists, and our development team leads and buyer staff have to manage the development of these increasingly software-driven components. Even if your organization still mostly makes or buys instructor-led training materials, Lean-Agile can still help your staff significantly improve during times when organizations are asked to do more with less resources.

With all the Lean-Agile content that is already in the world about Agile for software development, you might wonder what value a book about Lean-Agile for training could have. From what we have seen of Lean-Agile adoption in the training industry and from our experiences applying Lean-Agile to training, we still think there is a need for a Lean-Agile book that is specific to training because of the following:

  1. {color:#000;}During training industry conferences in recent years, we have talked to other attendees, and we found that most training buyers and producers represented at these conferences do not yet use Lean-Agile. From these anecdotal discussions, we have determined that there seems to be a slow increase year-over-year in the training industry’s use of Lean-Agile.

  1. {color:#000;}Some companies had training teams that supported software products where the software development team used Agile, but the training development team did not. This surprised us. One group even said they used Agile for their JavaScript developers who were making training products, but not for the training content developers.
  1. {color:#000;}More people may benefit from a direct translation of Lean-Agile principles to the training domain with specific examples from significant experience applying it in the training domain.
  1. {color:#000;}Training has already been broken down into a scalar that includes curricula, courses, learning objectives, and activities—this scalar varies outside the United States. Software often begins with a more generalized product breakdown structure of systems, subsystems, capabilities and features, so the Agile idea of requirements as epics and user stories (see glossary) helps to provide usable structure for software developers. Adapting Lean-Agile to training terminology can help more buyers and producers discover the benefits for themselves.
  1. {color:#000;}Experience has taught us many lessons while applying Lean-Agile to learning experience development that other practitioners and buyers may find of value. This is partly why we attend the industry conferences, to learn the lessons others are willing to share about their experience so we can all gain those lessons without each experiencing the challenges firsthand.

Similar to how Mary and Tom Poppendieck noticed a trend in software development way back in the early 2000s, we have also observed in years of training product development that “some methods which are still considered standard practice”[2] for developing learning interventions and training products “have long since been abandoned by other disciplines”.[2] Here, we emphasize project management approaches rather than instructional approaches, but we will explain more about that in a later section.

The value proposition for this book is similar to what Realtors tell you about selling your house, “Some buyers do not easily see past a color of paint they don’t like in a house you’re trying to sell, while a fewer number can envision it redone their way.” This book translates Lean-Agile into training terms, so you, as the audience, do not have to work as hard to connect the dots. It helps you to envision the newly painted house, so to speak. It can help us all speak the lexicon of Lean-Agile with a more common understanding of its applicability in training development.

[]4.3. A Brief Note to Chief Learning Officers (CLOs), VPs of HR or VPs of Engineering

For the executive responsible for buying training, take into consideration that:

  1. {color:#000;}You will need to assess how well potential suppliers can use Lean-Agile to fulfill their contract with you.

  1. {color:#000;}Training built using Lean-Agile should cost less for suppliers to create compared to traditional project approaches, so negotiate for lower prices after leaving the supplier room for a reasonable profit.
  1. {color:#000;}The more Lean-Agile principles you agree to allow and participate in, the higher potential savings and lower risk as a buyer.
  1. {color:#000;}Conversely, to the degree you insist on non-Agile behaviors in a hybrid approach, you increase the risk of higher production costs for the supplier and the consequent impact to what you may have to pay. At worst, costs may revert back to traditional approaches.

For the executive responsible for building training and learning using organic assets, consider this: If you have someone in your organization who wants to try incorporating Lean-Agile, we suggest a small experiment where you limit the downside potential and retain the potential of achieving a large upside.

If you are considering how Lean-Agile could help your organization, keep thinking this way. Consider that unless you have a persuasive and influential technician-level person with the skills in Lean-Agile already on your staff or in the organization, you will want to send him or her to some training on how to apply Lean-Agile or bring in an Lean-Agile coach to lead your first attempts. Or you may decide to engage a company like ours to build it for you using Lean-Agile.

Just as your internal quality staff will talk about the Deming Cycle of Plan, Do, Check, Adjust, you can also use this pattern to underwrite a series of experiments that allow trial and error to teach you what can work in your organization.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{background:rgba(0, 0, 0, 0);}.
p<{color:#000;}. Lean-Agile is not an all purpose “easy” button. It can help achieve greater results in building learning experiences, but it does not remove all uncertainty, randomness, and risk from the project. The human element that occasionally plagues organizational initiatives may hit this one too.

If you treat Lean-Agile as a magic solution and your staff doesn’t pull it off successfully on their first attempt, you may experience others blaming any failure on Lean-Agile and consequently reduce the likelihood of being able to apply Lean-Agile in your organization in the future.

It is better to keep expectations realistic when first installing Lean-Agile in your organization. Find someone who is skilled at leading change and that can build some enthusiasm about Lean-Agile and ask for their help experimenting with Lean-Agile. |

Another thing to be careful about—​If you have software people that know Agile and how software teams apply it with continuous integration, test driven development and other practices that are more specific to software development, be sure these people are capable of translating Agile principles into a training-perspective and adapting them to what works for training professionals. This caution also goes for the many consultants out there who primarily have software development experience with Agile. If you decide on a consultant to kick start your Lean-Agile initiative, be sure to find one that can translate software domain ideas and practices into training domain terms and practices to help your people adjust to the change with as little friction as possible. Some people will make the jump between domains, but when confronted with the normal stress of change, some may not get it as quickly if it is not explained within their own framework of experience.

If you have a potential Lean-Agile coach that has no exposure to the Theory of Constraints, then get them some TOC training. Train an in-house person as a Lean-Agile Coach to help retain the change over time. The importance of this increases with the number of Lean-Agile teams you plan to use.

Be sure your training team keeps some metrics of their historical productivity ratios, total cycle time per student hour or other appropriate metric so that you can discuss and extend any gains achieved. Use this historical metric, to compare and contrast against your new Lean-Agile initiative metric so you can demonstrate the gains and leverage the successes to get the support required to continue the Lean-Agile experiments on a larger scale.

Good luck.

[]4.4. Why Care about Lean-Agile as Buyers of Training Products and Services

In our experience, many organizations procuring instructional interventions tend to have someone on their staff who has high exposure to or even expertise in effective instructional approaches. Lean-Agile changes the way these training-aware people will interact on a project with a supplier or contractor in ways we will discuss later.

To take advantage of any improved cost and schedule performance for projects using Lean-Agile, there are things the buying organization must also do for the savings to be realized when a contractor or supplier uses Lean-Agile to build training products or services for them.

Our customers have asked questions before Lean-Agile projects and programs such as:

  1. {color:#000;}What do we have to do differently to accommodate your Lean-Agile approach?

  1. {color:#000;}How does your Lean-Agile approach change our process for procuring training products/services?
  1. {color:#000;}How does your Lean-Agile approach change what we’ll get or when we’ll get it?
  1. {color:#000;}How does your Lean-Agile approach impact our risk?
  1. {color:#000;}What do we tell our people to expect?
  1. {color:#000;}Can you provide a brief overview of how your Lean-Agile approach is different from ADDIE?

We will address answers to these typical questions in later sections.

[]4.5. Why Care about Lean-Agile as L&D Professionals

In our experience, many people leading training projects tend to have high skill levels and expertise in effective learning and development (L&D) approaches. In many projects, instructional/LX designers lead teams. Given the ends (learning outcomes), it is often appropriate that instructional/LX designers guide the means (design and development).

In our view, a growth opportunity exists for instructional/LX designers to keep up with the changing world. Rather than tending to focus all of your skills and knowledge in the workplace training domain, make room for monitoring ideas in other domains that can be applied to making instructional efforts better. For example, monitor the software development domain for practices we can use. Having interviewed, hired and worked closely with many instructional/LX designers, we have noticed that some seem more able and willing to visualize the processes for development of critical technology components of training solutions. These instructional/LX designers demonstrate more mindfulness to how the effectiveness in the design and development of technology components can impact their instructional purposes. Interestingly, the instructional/LX designers that demonstrate this also tend to request the more complicated challenges.

[]4.5.1. A Conceptual Framework for Categorizing Technology Components

Before we get deeper into how Lean-Agile can help us incorporate technology components, let’s take a moment to describe an analogy that can provide a conceptual framework for categorizing these technology components.

As you fly to your next course conduct or vacation destination, consider that the wings of the plane you’re in provide lift as they are propelled through the air (because the wing’s shape affects the speed of the airflow, which causes lower air pressure above the wing, or lift). Next, look out of your airplane window at that wing and note that the air flows over the wings, first passing the leading edge of the wing, traveling along the wing, and leaving at the trailing edge of the wing. Let’s use this idea to describe our relationship to technologies in training. If we look at training as the wing and technologies as the changing airflow, then we can say the following:

  1. {color:#000;}Leading Edge – A particular technology is leading edge if it is just now being experimented with and adopted by early-adopting training organizations or by projects with budgets sufficient to experiment with unproven technologies.

  1. {color:#000;}Mainstream – A technology midway over the wing is mainstream, and most are using it.
  1. {color:#000;}Trailing Edge – A technology at the trailing edge may still be used because its implementation methods have become so optimized and cost effective. Trailing-edge technology has been mastered and has an affordable ecosystem of tools that we use to apply it.

Within this framework, we can now state that Lean-Agile is leading edge for the training discipline, but is already mainstream for the software discipline.

[]4.5.2. L&D Professionals Applying Leading-Edge Technologies for Learning Experiences

Historically, our collective thinking about training contractors/suppliers/departments has been primarily as a service organization, where experienced people are brought in to analyze, design and develop instructional interventions using trailing-edge technology components of mostly language and images, and relying heavily on human-in-the-loop assistance of facilitators in all instructor-led or blended learning experiences. Sometimes, contractors/suppliers/departments even get to implementation by conducting blended instructor-led courses, and it is even rarer that they get an opportunity to evaluate the effectiveness of the intervention.

Now consider that as leading-edge training technologies get more complex and more coupled with software, the conceptualization and consequent labeling of training contractors/suppliers/departments will need to shift from mostly services to a combination of products and services. Technology-facilitated learning experiences, in particular, are quickly moving to more of a product than a service, particularly when an organization performs fewer human-facilitated course conducts for an increasingly global employee workforce. Gains in the market share of computer-facilitated learning experiences are more frequently found in the workplace and education. Rather than being merely a semantics exercise, a product-oriented viewpoint can lead to a more effective mental model about individual contributors and leaders in training organizations. Mental models are more valuable when they more closely align with the real challenges learning experience development teams face on a daily basis. Mental models make the merits of different methods of approaching project management for learning experience projects, methods such as Lean-Agile, more easily recognizable.

More often, we are seeing a convergence of training development and software development practices for complex workplace training products that include larger proportions of software-driven components. Therefore, the time has arrived to recognize that only using single-discipline project leadership by instructional/LX designers can increase project risk, and we need to take steps to mitigate that risk. For example, consider leading-edge technologies applied in our learning simulations (products), game-engine based training (products), courseware user interfaces (products), and mobile applications (products). Consequently if we continue to ask instructional/LX designers for buyers and producers to step up to the front of multi-disciplinary teams to ensure the integrated whole still achieves the intended learning objectives, then these instructional/LX designer leaders will need increasing familiarity with how to lead the work of technology specialists in support of the overarching learning outcomes. This is the growth opportunity for instructional/LX designers.

[]4.5.3. What Others Have Done Where Lean-Agile is Mainstream

Next, let’s look at how lessons learned in the software domain since Agile took over their world more than a decade ago is relevant to buyers and producers of learning experiences. We may benefit from their lessons learned.

Traditional software development used a sequential/stage-based approach often called waterfall. This approach generally used what they call functional decomposition, where an application or a “system” was broken into sub-components (e.g. user interface component, database component, etc.) which were then implemented in parallel and integrated late in the project lifecycle for testing and delivery.[3] For this parallel approach to work, the up-front requirements specifications had to be locked down with change control practices [3] called configuration management. The requirements specification was supposed to perfectly match what the customer wanted, too. Then, reality happened, unanticipated changes occurred and late changes occurred anyway. Then, issues surfaced late in the project when all the decomposed components were integrated back into a single whole because the requirements had changed, creating much rework. This may sound familiar with training projects.

Today, many Agile software development projects decompose the system being developed into manageable sets of requirements, often called “features” or “user stories”. The software product or system is segmented into increments of customer-valued functionality, and development work is organized around building these increments.

The software discipline has largely moved away from many components built separately in large batches and holding integration and testing until the end. Instead, they have moved towards building working feature-sets in smaller batches, integrating them and testing them earlier in the project lifecycle. Their discipline has now provided evidence of reducing uncertainty (management reads as risk), detecting defects earlier, resulting in less rework, and delivering early capabilities to customers (as opposed to waiting longer for a full-featured product).

[]4.5.4. Back to the Future – Increasing Technology in Training

Traditionally, much workplace courseware has been developed in serial, or lesson order, and depending on how far back we go, it was mostly text and two dimensional graphic art. Although not the main point of this book, also consider that most of our text has been authored, read encoded, into proprietary software application formats which adds risk and adds dependencies, tying us to the vendor’s decisions about commercial toolchain lifecycles and support. See Markup Technologies.

Historically for low-technology training with the work products being mostly text and two dimensional (2D) graphic art content, integration of such text and 2D art components was not really an issue, and the training profession did not then see the need to include the term integration in our training domain lexicon. See the glossary for what we mean by integration. Even as time went on, when the majority of training products were instructional support packages for trainers/facilitators to implement a course, we had no apparent need for other approaches. Sequential ADDIE waterfall worked fine.

Fast forward 40+ years with the accelerating pace of technology change and its impact on training. In addition to the typical content, technology often adds functionality that supports learning experiences. As buyers/customers want newer technologies blended into training, these technology components have tended to be built by specialists in parallel to the main instructional development effort. These technology components then need to be integrated into the overall learning experience with (a) traditional content checks and now (b) functionality testing to ensure the integrated components function together as expected and support the intended learner experience. Though training staff are not yet habitually using engineering terminology to describe their work, working in cross-functional teams means collaborating with people from other disciplines who use this terminology. Disciplines such as Systems Engineering, and Software Engineering already have many practitioners using Lean-Agile. This is one reason why L&D professionals should care about Lean-Agile.

In the training industry, before the recent apparent dominance of rapid development tools like Captivate, Lectora, Storyline, various software as a service (SaaS) authoring tools, et cetera, teams included specialists who built the “content player” if you will—​sort of like Apple built the iTunes player and the artists build the musical content. Before rapid development tools, instructional content was authored in various ways, but someone on the team had to “integrate” it all into a Sharable Content Object Reference Model (SCORM) bundle that included the content player and the content. When we want to go beyond the relatively few options offered by such rapid development tools for learning experiences, we will need to include other disciplines on our development teams. This is another reason why L&D professionals should care about Lean-Agile.

There are many factors that influence which technologies make it into our learning experiences. One constraint that will likely continue is that business leaders tend to look for evidence that training interventions are having an impact on individual performance and organizational objectives before deciding how many resources to allocate to training procurement and production. Pausing to look for proof of effectiveness is similar to playing the children’s game red-light, green-light with training budgets. This implies that we have to know the following:

  1. {color:#000;}How to manage projects that incorporate technology components in learning experiences

  1. {color:#000;}How to work with those experts that help create our desired technology components
  1. {color:#000;}How to run projects quickly enough so that learning interventions are timely to the current business needs

This need to increase our probability of success when using technology components in learning experiences is another reason why L&D professionals should care about Lean-Agile.

As learning and development continues to grow beyond low-effectiveness “telling is training” to learning by doing, technical requirements will continue to push us beyond our traditional comfort zones for managing these design and development projects. We say that Lean-Agile can help you succeed.

Buyers of training may ask producers to go beyond basic, trailing-edge technologies for blended components, eLearning, mobile learning and game-engine based training. They may desire additional leading-edge technology components that often have their own detailed design and development pipelines. This means that producers will need enough familiarity with the production processes to be sure the leading-edge technology components that support learning experiences are on track as well as the learning content.

Lean-Agile can help successfully manage these new technology components to successful completion, so we can build a timely, cohesive, whole learning experience that achieves the intended worksite performance goals.

Consider another external influence. The public schools in the United Kingdom are now requiring all students to be literate in software coding. Should this idea spread, then this changes both the training industry’s future contributors and all of our future customers, and their expectations. Some exposure to Lean-Agile may come at high-school level going forward, making it easier for training organizations to have the talent to deal with all the new functionality requirements (players, interactivity, simulation, etc.) that are desired in addition to the more traditional focus on content. This is another reason that L&D professionals should care about Lean-Agile.

Earlier training technologies have been full of production surprises late in their lifecycle and testing was sometimes fraught with late-night panic that the deadlines would be missed, not because of the solid instructional components, but due to the new, and often more fragile, or at least less well understood, leading-edge technology components. The team lead with an instructional/LX design background may not be sure what the problems and issues are and why the specialists supporting their project can’t solve the issues without delaying the learning experiences from working as expected by the deadline. Some technology specialists also have challenges articulating their process steps, progress and the significance of challenges to instructional/LX design leads as people that don’t have a hands-on level of understanding in all the details needed to implement this particular new technology component.

Perhaps that last comment sounds familiar to L&D professionals working with technology specialists. It may also work the other direction and sound familiar to L&D professionals struggling occasionally with customers or leadership that understands the big picture, but who are not as deeply fluent at the hands-on level with all the details that are involved with complicated blended learning experience development. We want to reinforce that they are knowledgeable, and the technical staff is expected to have fluency at the finer-grain level to be able to implement learning solutions that integrate technology.

How has expecting simple answers to the complex problems of our supporting specialists worked out so far? Lean-Agile helps address the complexities involved in technology-enabled learning experiences because Lean-Agile deals well with complex systems.

[]4.5.5. Examples of Technology in Learning

The following is a partial list of technologies demonstrated at recent trade shows and by projects we have performed that require teams that include people with specialties that go beyond instructional/LX design. The effort of leading these projects successfully and reducing the uncertainty that comes with some of these components is enhanced with Lean-Agile.

  1. {color:#000;}Interactive video segments

  1. {color:#000;}Branching video stories
  1. {color:#000;}Deterministic branching decision scenarios to five or more levels
  1. {color:#000;}Non-linear editing of branching scripts for non-player characters (both development and review)
  1. {color:#000;}Complex animation
  1. {color:#000;}Complex interactions that are beyond the small library of widgets in current rapid authoring tools
  1. {color:#000;}Virtual classroom breakout session materials
  1. {color:#000;}eLearning, interactive multimedia instruction, web-based training or other names for computer-facilitated learning
  1. {color:#000;}Mobile learning native apps, web apps and hybrid apps
  1. {color:#000;}3D avatar mentor characters that help facilitate the learning experience
  1. {color:#000;}Conformance to the eXperience API, xAPI (formerly Tin Cup API)
  1. {color:#000;}Innovative uses of xAPI data to measure job site performance and learning transfer
  1. {color:#000;}Accessibility regulations that insist on equivalent facilitation for disabled learners
  1. {color:#000;}Game engine user interfaces
  1. {color:#000;}3D models of people, equipment and facilities
  1. {color:#000;}Process simulations with user interfaces
  1. {color:#000;}Soft-skills simulations with user interfaces
  1. {color:#000;}Probabilistic (stochastic) discrete event simulations in support of learning
  1. {color:#000;}Systems dynamics simulations (see The Thinking Effect, by Michael Vaughan for more details.)
  1. {color:#000;}Software models for realistic system simulator behaviors for operations and maintenance training
  1. {color:#000;}Immersive environments used for virtual skill assessment based on doing rather than knowing
  1. {color:#000;}User interfaces with user controls so the learner/player is in control of the experience
  1. {color:#000;}Blended designs that require the instructor/facilitator interface to let the facilitator stop or pause the technology component briefly to address observed learner confusion
  1. {color:#000;}Embedded training for simulator operation and maintenance that is built into the simulator as tutorial levels rather than as traditional stand alone training
  1. {color:#000;}Networked simulators
  1. {color:#000;}Adaptive learning systems based on an adaptive engine that compares learner performance against predictions and automatically adjusts what comes next based on performance
  1. {color:#000;}Holographic visuals in simulators

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Many of these technology components have design and development process pipelines that are much more similar with software pipelines than with traditional training pipelines. This is one of the reasons for this book, because Lean-Agile works very well for these training pipelines.|

Due to their complexity, many of these technology components take longer than a single two-week iteration to complete. This means we need to know where reasonable break points exist to divide the work into smaller work items that can be completed within a single iteration to show progress.

Occasionally, technology components require discovering how to implement, especially those technologies closest to the leading edge.

Sometimes, we put our head in the sand and treat component creation as black box components that go to specialists with vague storyboard descriptions or requirements specs. With this approach, there is almost no interaction until the specialist creators are done. This approach means we can be surprised at the end of their estimated allotment of hours with the binary situation of problems (0) or success (1). Some will prefer to accept this increased project risk, while others will use Lean-Agile to mitigate the risk.

We propose that you consider managing the entire design and development of technology components visually on a Lean-Agile Kanban board, so you have insight on their progress earlier in their life cycle. Schedule internal iteration reviews/demos of working chunks with you as the customer to help you reduce the uncertainty to the rest of the learning experience.

For a more detailed view of where the future of training may be going, see Learning by Doing, by Clark Aldrich, and the realistic training simulation approaches described by Michael Vaughan in his book The Thinking Effect: Rethinking Thinking to Create Great Leaders and the New Value Worker.

[]4.5.6. Lean-Agile as a Means to the Ends

Lean-Agile visual Kanban boards can help buyers and producers see their workflow process steps and see the progress of work items in process. It can also help L&D professionals gain more awareness of what is involved in making X or Y leading-edge technology components in support of learning experiences. This can increase their capacity to realistically assess feasibility, report progress to leadership and see when issues are occurring, which offers opportunities to bring in additional resources earlier.

Lean-Agile provides benefits to modern buyers and producers of training products:

  1. {color:#000;}A way to improve our own cycle times

  1. {color:#000;}A means of gaining visibility into complex technology components that are in process
  1. {color:#000;}Visual signals for when “get well” interventions are needed
  1. {color:#000;}An environment that provides more information without micromanagement
  1. {color:#000;}Improved chances of leading successful projects
  1. {color:#000;}Smaller chunks of working learning experiences are produced earlier in the project
  1. {color:#000;}Reduced uncertainty from the technologies injected into the learning experiences

Most software developers already know Agile, and some may know Lean too. As we all use Lean-Agile to develop both the instructional activities and the supporting technologies for interactivity levels and engagement, we can get to the delivery date with less panic and more trust that our instructional/LX design will accomplish its goals.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. None of this reduces the hard work of instructional/LX designers. Developing effective learning will continue to be challenging.|

So, this book attempts to show how we can use Lean-Agile in learning experience development to reduce risk and uncertainty, to build learning experiences, even with new and complex technology components, in small working increments as we go rather than waiting until later in the process to see large batches of scope and discover whether or not they work.

[]5. Setting the Stage[]

[]5.1. Readers Familiar with Lean-Agile in Software can Skip this Chapter

If you already have exposure to Lean-Agile from another field such as software development, then you can skip this chapter and go on to how we propose adapting it to learning experience development.

This chapter is designed to bring those new to Lean-Agile to a common baseline. This chapter explains how we got to today in Agile and Lean from the 1980s until now.

[]5.2. A Brief History of Applying Ideas from Other Domains

It may help to briefly recap how others have successfully applied ideas from one domain to another before beginning our proposal to apply ideas from other domains to learning experience development.

Successful examples include the following:

  1. {color:#000;}The Gutenberg printing press applied ideas from the wine press (an entirely different domain) to have a printing press apply ink to paper [4]

  1. {color:#000;}Medical Doctors are starting to apply ideas pilots use by using checklists in aircraft operations, by adopting surgical checklists [5]
  1. {color:#000;}To describe a point of interaction between two technologies, systems or components, engineers use the term physical interfaces.

–One example of a familiar physical interface is the telephone dial tone. This physical interface was originally intended for one phone to connect with another phone. This interface has remained unchanged much longer than the technologies it connects. On the consumer side of the interface, our parents and grandparents remember rotary dial phones and how long it took to simply dial a number. Later came fax machines, touch tone phones and mobile phones. On the supplier side of the dial tone interface are telecommunications exchanges which have progressed from human operators to analog, and now to digital electronic components that switch circuits to connect telephone calls between users.

  1. {color:#000;}Software engineers have used this interface concept from the physical domain and applied it in the domain of software applications. For example, object interfaces in object-oriented programming allow other objects to only access the so-called public interface; the implementation details are hidden as private or “encapsulated” so those details can change as needed without impacting all the objects they serve.

  1. {color:#000;}There are also interfaces not specific to software development. Because dependencies can lead to cascading failures between coupled things, interfaces help to decouple components of larger, more complex systems allowing us to improve how each component works (like rotary dial to digital touch dial) without having to remake the entire system for every small change. Applying this concept of decoupling components across domains has already proven successful. Specific to object-oriented software, decoupling components of the system can also be accomplished using abstract classes and minimal public interfaces that do not ‘know’ much about other objects.
  1. {color:#000;}Edward Deming helped apply statistical methods he learned in the domain of agriculture to the domain of automotive manufacturing.
  1. {color:#000;}Originally developed for telecommunications, Information Theory has been applied in many other areas, with particular success in code breaking, or cryptography.
  1. {color:#000;}Latin in the West and Chinese in the East have both been applied in the formation of other languages in other cultures. The Japanese adapted Chinese characters into their own language. Latin roots are still evident in many Western languages.
  1. {color:#000;}Accretion is another idea that was borrowed across domains. Semiconductor manufacturing applied the ideas from flakes of snow accumulating little by little for chemical vapor deposition techniques to accumulate other chemicals on a substrate wafer, used in making computer chips. The idea of accretion has also led to application in so-called additive manufacturing where 3D printers are used to make physical objects one layer at a time.
  1. {color:#000;}Redundancy, or having reserves, functions as a buffer against volatility in many domains. Living organisms exhibit redundancy. For example people have two kidneys. You can donate one kidney, but that makes you more fragile. Adopting reserves allows financial institutions, and families, to survive unexpected shocks from their environment. The military uses reserve forces to capitalize on success in offensive battle or to reinforce at key points during defensive operations. Human-designed systems intended to be robust and “antifragile”, to use a term introduced in the book Antifragile – Things that Gain from Disorder by Nassim Nicholas Taleb, adopt this idea. For example, the U.S. Army’s Blackhawk helicopter has redundancy in its systems which have allowed some of these aircraft to fly their crews and passengers back to safety after receiving much damage from hostile fire.
  1. {color:#000;}The book, Where Good Ideas Come From, by Steve Johnson, describes how a medical doctor in the 1870s took a walk through the local zoo and saw chicken incubators and applied the idea from the animal zoo to the human maternity ward to help improve the chances for human babies to survive. This doctor’s statistical analysis is what helped the transfer from one domain to the other succeed.

These examples are sufficient to make the point that recombination of ideas, concepts and components from various domains is how much of our human advancement in technology has occurred. We contend that we can continue to do so by applying ideas from other domains to the domain of workplace training and learning experience development.

[]5.3. A Brief History of Agile

The discipline of software engineering brought us Agile as an improvement over what they had used before. Since the 1970s, older methods have been used for software development. The software development life cycle (SDLC) is one of these older methods.

The SDLC included a linear process of sequential phases with the work flowing down through the process steps like a waterfall, thus the nickname the waterfall method.

Some models show these phases as requirements, design, implementation, verification and maintenance.[6]

The older waterfall methods expected that the customer knew exactly what they wanted at the beginning of a project with their requirements document specifying what they wanted. Experience has proven that what customer’s want has tended to change during a project. Over time and from many projects, software developers had experiences that indicate that customers do not always know exactly what they want up front, that they often change their requirements during the project. Because software projects can take so long, sometimes even the customer’s business changes during long projects. This experiential data from software development also matches human nature at not often getting work done exactly right the first time. Change happens constantly. This may sound familiar in learning experience development, too.

By the late 1990s, many software development practitioners felt these older methods were too slow, too strict and encumbered by too many document artifacts. Experimenters, like Jeff Sutherland and others, started developing alternatives to the heavy-weight waterfall SDLC process. These experimenters sought lighter-weight methods. Then in 2001, a group of software development practitioners got together in Snowbird, Utah to discuss light-weight methods. This group worked together to write what they called the Agile Manifesto for Software Development in an effort to influence other practitioners of Software Development. The manifesto writers included practitioners of various software development methods such as Adaptive Software Development (ASD), Agile Unified Process (AUP), Crystal Clear Methods, Disciplined Agile Delivery (DAD), Dynamic Systems Development Method (DSDM), Extreme Programming (XP), Feature-Driven Development (FDD), Lean software development, Kanban, Scrum and Scrumban. They wanted better options than the SDLC (often called waterfall, documentation-driven, or plan-driven). They named themselves the Agile Alliance.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. It is beyond the scope of this book about training to describe each of these light-weight methods of software development.|

The Agile Alliance also wrote their principles of Agile for software development. It is these principles that we will apply to training.

The principles indicate that instead of focusing as much effort on up-front planning and detailed requirements, Agile methods place more emphasis on:

  1. {color:#000;}Continual planning that gets distributed across the project

  1. {color:#000;}Cross-functional teams rather than handoffs between departments of specialists
  1. {color:#000;}Team collaboration
  1. {color:#000;}Emergent design
  1. {color:#000;}Functionality testing early and often
  1. {color:#000;}Frequent delivery of working software in short iterations

Agile is an umbrella term that describes principles that are derived into various light-weight methods. Software developers as a community have many discussions and arguments about Agile. It is a spirited discussion that provides good insights. Other than the Agile Manifesto, there is no consensus on a definition of the term Agile for software. We propose a definition for the purposes of this book in the glossary. Agile can mean any group of development methods that apply the Agile principles. Software practitioners also have good discussions about partial versus full implementations of Agile. We will address that later. For the purposes of this book, we will stick to the idea that if you are applying Agile principles in your specific context, then it is okay if your specific methods vary. Because we are applying Agile principles to learning experience development, our methods will necessarily vary somewhat from popular software development Agile methods as we experiment and adapt.

Managers may perceive Agile as anti-process or anti-methodology. Agile is not anti-process or anti-methodology, but seeks lighter-weight processes over heavy-weight processes. Planning is okay in Agile too. Agile recognizes that plans have limits in complex situations. We discuss this in more detail later. Artifacts that document process are okay in Agile. Agile tends to focus on light-weight, meaning short for documents, rather than long. Lighter-weight encourages actual use. By counter example, heavy-weight plans use lots of boiler plate text, are ill-maintained and consequently teams rarely use them.

Since the 2001 Agile Manifesto, Agile has spread to many more groups making software. One Agile practitioner, David Andersen, applied lessons from other industries as he continued to experiment and adapt. David adapted Lean thinking from manufacturing into software product development, naming his method Kanban. Lean principles increasingly have been integrated into Agile. We will discuss Lean more in the next section A Brief History of Lean.

The rapid rate of adoption of Agile methods for software development has resulted in many software developers being aware of and using various Agile methods. In contrast, awareness of Agile approaches in the learning and development discipline is just beginning. Usage of Agile is also just beginning. The improvements Agile brings to project results such as cost, schedule and customer satisfaction have begun to catch the attention of other industries. Even the U.S. Federal Government is beginning to allow Agile development methods in its procurement of software products. Agile now has certifications from the Project Management Institute.

[]5.4. A Brief History of Lean

Lean comes from the world of car manufacturing after World War II. Taiichi Ohno is considered to be the father of the Toyota Production System, which became Lean Manufacturing in the U.S.[7] Ohno and Toyota looked at the Ford Motor Company and U.S. supermarkets for inspiration [8], perhaps seeking principles, and combined what they learned from the United States with their own ideas and formed their own set of methods which they called the Toyota Production System.[8] Toyota developed a way of making cars with less inventory because they could not compete head to head with the huge inventories of American car companies given the difficult economics of post-war Japan. What Toyota learned to do with less resulted in their production process. Toyota called their method the Toyota Production System.[9] [10] Their production system was described as “Lean Manufacturing.”

The term lean thinking [11] was coined by James P. Womack and Daniel T. Jones to capture the essence of their in-depth study of Toyota’s fabled Toyota Production System.[12] Lean thinking is a new way of thinking any activity and seeing the waste inadvertently generated by the way the process is organized by focusing on the concepts of:

  1. {color:#000;}Value,

  1. {color:#000;}Value streams,
  1. {color:#000;}Flow,
  1. {color:#000;}Pull,
  1. {color:#000;}Perfection.[12]

These are the five main Lean principles.

During the 1990s, 2000s and 2010s, Toyota continued to be one of the largest auto manufacturers in the world. Their success has drawn others to look to Toyota as an example of Lean Manufacturing and Lean Thinking.

Lean principles have been applied in other industries and product development successfully. See the 2011 book, Lean for System Engineering, by Bohdan W. Oppenheim for a complete example of applying Lean to another product development domain. There are many written works about Lean now.

Note|<{color:#000;background:rgba(0, 0, 0, 0);}. There is a large amount of information on the Internet about Lean and the in-depth history of Lean is beyond the scope of this book.

[]5.5. A Brief History of the Theory of Constraints

Dr. Eliyahu M. Goldratt developed his Theory of Constraints and described it in his popular book, The Goal, by Eliyahu Goldratt and Jeff Cox. Goldratt later credited Toyota and Lean as his foundation for the Theory of Constraints.

To help organizations profitably succeed, the Theory of Constraints aims at increasing throughput, reducing inventory and reducing operating expenses.

Training buyers and producers may not be accustomed to thinking of partially completed learning experience chunks as inventory. So, the shift in thinking takes some adjustment to see "inventory" and adopt another domain’s view, from manufacturing over 25+ years, that inventory is bad.

The Theory of Constraints is a way of identifying the main limiting constraint, or bottleneck, in a process and then systematically improving that constraint.

The Theory of Constraints aims to help us answer five big questions:

  1. {color:#000;}“What needs to change first?” (the weakest link in the process ‘chain’)

  1. {color:#000;}“What should it be changed to? What is the To-Be state?”
  1. {color:#000;}“What actions will cause the change?”
  1. {color:#000;}“Did the change improve throughput, inventory or operating expenses?”
  1. {color:#000;}“What needs to change next?” (the next weakest link)

The Theory of Constraints also accounts for complex adaptive systems with many linked activities. It provides the following:

  1. {color:#000;}A way to identify and eliminate constraints

  1. {color:#000;}Thinking tools for problem solving (root cause analysis tree diagrams, 5 whys, etc.)
  1. {color:#000;}A way to prioritize improvement actions on one main constraint at a time
  1. {color:#000;}Compatibility with Lean and Agile
  1. {color:#000;}Reduction or elimination of bottlenecks resulting in less work in process (WIP)
  1. {color:#000;}Improvement of overall organizational profit, rather than sub-optimizing at a lower level

The Theory of Constraints consists of five steps:

  1. {color:#000;}Identify the constraint

  1. {color:#000;}Exploit. Improve what you can with what you’ve got aiming to increase the throughput of the constraint
  1. {color:#000;}Subordinate. Subordinate the rest of the process to the constraint or bottleneck. Don’t let them go faster.
  1. {color:#000;}Elevate. Do what you can to improve it further.
  1. {color:#000;}Repeat. Look for the next bottleneck. Kaizen. Continual improvement.

Many organizations use Goldratt’s Theory of Constraints to focus Lean efforts on the priorities that matter to the business or enterprise.

For more information, contact The Theory of Constraints Institute (http://www.tocinstitute.org).

[]5.6. A Brief History of the PDCA Cycle

Popularized by Deming and adopted by Toyota in the 1950s, the PDCA cycle is memorable method for problem solving towards continual improvement that uses four steps:

  1. {color:#000;}Plan

  1. {color:#000;}Do
  1. {color:#000;}Check
  1. {color:#000;}Adjust (also called Act)

With a little more detail on this pass:

  1. {color:#000;}Plan. Sense an issue, define the problem, plan a small experiment, change or test, aimed at an objective like improvement, often form a testable prediction called a hypothesis, identify the who, what, when, where, why

  1. {color:#000;}Do. Do the small experiment, change or test
  1. {color:#000;}Check. Observe actual results and then compare to any predictions to see what the results tell us
  1. {color:#000;}Adjust. Modify next actions based on what was learned from the feedback of the small experiment

The PDCA cycle has become the basis for evidence-based decision making in many organizations.

|={color:#000;background:rgba(0, 0, 0, 0);}. Tip|<{color:#000;background:rgba(0, 0, 0, 0);}. The original “A” was act in the Toyota version, but many use the alternate “A” word adjust. We think the word adjust better describes the action of modification based upon feedback, so we use adjust in this book. Use what you want for the A as long as you cycle through PDCA often enough to improve. The PDCA cycle is a simple way to use experimentation to continually improve your processes.|

The PDCA cycle came to us from the domain of quality. PDCA was made popular by Dr. W. Edwards Deming, who is considered by many to be the father of modern quality control.[13]

During his lectures in Japan in the early 1950s, [13] Deming got the Japanese interested in his methods of quality control.[14]

Toyota calls it the PDCA cycle. It also became known as the Deming Cycle; although, Deming actually called it the Shewhart cycle after his mentor, Walter A. Shewhart, the originator of Statistical Process Control (SPC). You may hear it referred to by any of these names.

The U.S. War Department recognized Deming’s expertise in statistics and asked him to go to Japan in the late 1940s to study agricultural production and help with related problems in this war-damaged nation. In the 1950s, General Douglas MacArthur asked him to assist Japan with their census. Deming taught hundreds of Japanese engineers and industrial managers his ideas to help correct their quality problems.

The Japanese embraced Deming’s quality concepts and PDCA and used it to go from worst in quality in the 1950s to best in quality by the 1980s in multiple products. Deming had considerable influence in Japan. Their highest award for quality is called the Deming Prize. The rise of Japanese manufacturing from the 1950s to the 2010s was influenced by Deming’s efforts and his PDCA cycle which became widely used for improvement. After Japanese companies used Deming’s ideas to such great effect, American businesses took note, and many began to adopt Deming’s quality concepts in the 1980s.

All of this success in this domain has inspired the application of Deming’s PDCA cycle in other domains.

[]5.7. Visualizing Workflow

Developing learning experiences is knowledge work. Knowledge work makes it hard to observe work in process. Toyota initially developed Kanban for manufacturing, organizing the factory to visually see the process more clearly. Kanban allows Toyota to see components queued up for various process steps, and to establish pull with downstream process steps requesting resupplying when they need it. Unlike with knowledge work one can easily see the manufacturing inventory of work in process by walking around a factory floor. See the blog entry [15] of the original creator of Kanban for knowledge work for more details on how he created Kanban from 2004-2006. David also applied the Theory of Constraints at Microsoft. David presented his work at various conferences. Andersen’s book Kanban: Successful Evolutionary Change for your Technology Business, identifies the principles he discovered. An important principle that Andersen’s Kanban includes is borrowed from Lean and is called visualize the workflow. After learning about David Andersen’s work, we began to experiment with using Kanban boards to visualize our workflow for making learning experiences.

When working alone, “workflow” is an easy thing to visualize. Some people still use an inbox and an outbox on a real desk, so let’s use that scenario to illustrate a one-person workflow in the following figure. If you first have to identify all the tasks that must be accomplished and put them into the inbox, then this comes first.

Figure 1. One-Person Workflow

Next, combine three to seven people on a team and their workflow may look different. Let’s say you’ve just gotten an existing training analysis, and your team is asked to build learning experiences appropriate to that analysis. Your colocated team gets a group work area with a white board nearby. One team member draws the following columns for the workflow steps, and the team lists the work items that have to be done on sticky notes and puts them on the board in the in box, which we’ll now call a “To Do List” or backlog. Your workflow is now visible to the entire team.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. These Kanban columns are only illustrative. They do not need to be the columns used for your Kanban. It can be as simple or as complex as you need for your context. Your columns may vary.|

Figure 2. Whiteboard Workflow Columns Example

Next, the instructional/LX designers each pull one or two work item cards into the design column and those instructional/LX designers begin to work on those few items.

Figure 3. Next Step, Tracking the Progress of the Work In Process

Work continues for this step until those work items are done. Then, you move the work item cards to the next step/column of the workflow, and the opening in the Design column allows two more work item cards to be pulled into that column.

Figure 4. Seeing the Work Move Through the Workflow

Determine the workflow steps for your team’s process and then you’re ready to make a Kanban board.

Rather than the team lead updating some report or spreadsheet about the team’s progress, the team members each move the cards they take accountability for, and the entire team can see the state of the current work in process anytime, 24-hours per day. Their management and the Buyer sees the Kanban board snapshot at least once per day during the daily standup meeting.

Visualizing workflow is not complicated. Although we will discuss more specifics of Lean-Agile later, this is sufficient for understanding what is meant by visualizing workflow for the time being.

[]5.8. Trello as a Virtual Kanban Tool

Ideally your team is colocated and can use a physical whiteboard. However, for geographically dispersed teams, a physical whiteboard is not ideal for everyone involved to see the team’s progress at anytime. In these situations, a virtual Kanban board can help.

You can find various suppliers of virtual Kanban boards.

By way of example, let’s briefly look at one so you can see that the use of a virtual Kanban board is nearly the same as for a physical Kanban board.

For this example, we’ll use Trello. (www.trello.com). Their online visual project management system can easily be adapted to make Kanban boards.

So what might a version of that team whiteboard look like? Here is part of the Trello version, designed exactly like the physical whiteboard example shown in the previous section.

Figure 5. Trello Version of the Earlier Whiteboard Example

Find a supplier that works for your needs.

[]5.9. Lean-Agile is a Big Skill that Combines Middle Skills

Clark Aldrich uses a framework [16] for addressing larger scale competencies in learning simulations. Clark also wrote an excellent book in which he also discusses big skills, called The Complete Guide to Simulations and Serious Games. His scalar distinguishes between:

  1. {color:#000;}Big Skills (highest value)

  1. {color:#000;}Middle Skills
  1. {color:#000;}Actions (doing at the tactical level)

We propose that Aldrich’s framework also applies to Lean-Agile. We posit that the big skills he identifies, like leadership and relationship management, also include Lean-Agile.

Big skills require what Aldrich calls bundles of middle skills. His framework description aptly restates what most people have learned from their own life experiences, that knowing is insufficient, and that big skills have to be practiced to provide their full potential value. Learning to lead others requires a necessary knowledge base, but it also requires much practice to become effective. Note that ensigns, second lieutenants, and first-time supervisors of many job titles are often portrayed poorly in many books and movies. Consider such people in your own experience. Surely there is truth in the idea that practice is required to hone big skills.

We propose that Lean-Agile is a big skill made up of at least the following middle skills:

  1. {color:#000;}Application of Agile principles

  1. {color:#000;}Application of Lean principles
  1. {color:#000;}Application of Queuing Theory
  1. {color:#000;}Application of the Theory of Constraints
  1. {color:#000;}Systems thinking within complex adaptive systems
  1. {color:#000;}Identifying and using leverage points and feedback loops
  1. {color:#000;}Plan, Do, Check, Adjust (PDCA)
  1. {color:#000;}Negotiation
  1. {color:#000;}Expectation management
  1. {color:#000;}Change management

It also requires other big skills that Clark identified, like:

  1. {color:#000;}Organizational, Team, and Self Leadership

  1. {color:#000;}Relationship management

This is why learning to apply it well may seem as difficult as learning leadership. There are many middle skills and actions that make up the big skill. Trying to learn all of the big skill’s applicable middle skills and actions at once can be just as overwhelming as trying to learn leadership in a single session. Note that most leadership development programs do not attempt a one-shot approach either.

That Lean-Agile is a big skill is why we also suggest that simply bringing in an Lean-Agile consultant for a short duration will not result in an organization that performs Lean-Agile well as soon as the consultant completes the engagement and departs. Because Lean-Agile is a big skill, we suggest having a Lean-Agile coach on staff for a significant period of time as your organization learns to apply Lean-Agile to buying or making training products.

According to Aldrich, we have to build up capabilities with actions and middle skills before we will gain repeatable success with the big skills.

Unfortunately, much of the literature about Agile, Lean, the Theory of Constraints, systems thinking, complex adaptive systems, and PDCA focuses on the actions and middle skills, presenting long lists of knowledge, skills, and attitudes with little focus on how to combine these middle skills and actions into bundles and wield them well together. Some of that comes with practice. We attempt to describe some of that to the degree that a book can transfer such things.

However, knowledge is necessary and insufficient. We have to gain the wisdom to wield Lean-Agile with the appropriate timing and magnitude through our own experiences.

So this book will, of necessity, list Lean-Agile middle skills, and some actions, but the key to using Lean-Agile well is to practice it enough to learn to use bundles of this middle layer of knowledge, skills, and attitudes.

Like learning leadership, or skills in the psycho-motor domain such as golf or the piano, knowledge, skills, and attitudes grow through accretion rather than suddenly from reading a single book. We have to pay the price to gain any valuable competency, and this human pattern holds for Lean-Agile too.

Learning and practicing is within your control. Even if you need to start on personal projects as a small practice environment rather than organizational projects/programs for your employer. Set some modest goals (PLAN). Start with the least change required (DO). Agile Kanban requires little change to existing processes. Begin with it to gradually build experience in Agile. See what the impact is (CHECK) and improve how you go about it (ADJUST). Some degree of improvisation can help as you better discover what works for you.

|={color:#000;background:rgba(0, 0, 0, 0);}. Caution|<{color:#000;background:rgba(0, 0, 0, 0);}. We do not mean Kanban as described and used in manufacturing, but Kanban as David Andersen developed it for software teams at Microsoft and describes it in his written works for knowledge workers.|

For a future iteration, add a Lean-Agile Coach and begin to expand Lean-Agile expertise to other people in your organization. In another iteration, continue to practice and apply by adding to your efforts with Scrum or another Agile method you prefer. Use Lean initially by removing wastes, but don’t stop applying Lean only with waste reduction when it can help you increase value. Combine Lean with the Theory of Constraints to focus improvement efforts on increasing value.

Apply the scientific method with the PDCA cycle to learn from cycles of forming testable hypotheses, testing them, and gathering data for what worked and what did not go as expected. Use that data to drive your improvements in regular intervals. But, like in golf, if you focus too much on all those little things while in mid-swing, you may not put the ball where you intend. Lean-Agile initiatives require effective leadership to succeed. Clark Aldrich identifies leadership as one of those “big skills” that requires using bundles of these middle skills to perform effectively. Based on Aldrich’s criteria for big skills, we suggest that Lean-Agile is also a big skill.

Developing big skills requires participants to experience cycles of frustration and resolution.[17]

— Clark Aldrich

This empirical approach tends to yield results more fit-to-purpose than simply adopting someone else’s tenets that tend to be bundled with their specific context, constraints and assumptions. Using the theories of others can be a good way to help jump start your own efforts. Because it is easy to visualize, let’s go back to the children’s game of red-light, green-light. Start your Lean-Agile efforts with red-light, pausing to reduce your risks by treating what anyone else postulates as your beginning hypotheses, then design your own experiments to test whether these predictions hold up for your specific context, constraints and assumptions. If the data indicate it fits your purposes, then signal green-light for your organization’s next steps in allocating effort and resources to implement Lean-Agile.

[]5.10. Your Organization May Vary

Training organizations vary significantly. Some types include:

  1. {color:#000;}A group which buys learning experiences for their organization

  1. {color:#000;}A training group supporting hardware product development or manufacturing of systems
  1. {color:#000;}A L&D group supporting a human resources learning and development organization or sales training
  1. {color:#000;}A training group supporting software product development
  1. {color:#000;}A training group supporting a services company or organization
  1. {color:#000;}A training group selling training products to businesses and governments
  1. {color:#000;}Volunteers in an open source project

How you adapt Lean-Agile to your organization depends on your organization in addition to other constraints. The following sections provide some suggestions.

[]5.10.1. A Buyer Group which Procures Learning Experiences for their Organization

Some groups buy third-party training products and services for their organization to help the organizational members better perform and help the organization meet its goals. Perhaps buyer groups may not yet have heard of Lean-Agile as it applies to training. Benefits to buyers of Lean-Agile include:

Buyer Benefits

  1. {color:#000;}Visibility

–All stakeholders are provided with maximum visibility into project progress

–Less surprises, less “trust us” and more “show me”

–Use actual learning experience increments as the measure of progress with both buyer’s internal stakeholders and the contractor/supplier

–Frequent delivery of chunks, or increments, of the total learning experience

–Earlier and improved buyer inspections and evaluations of working learning experience increment deliveries

–Promotes transparency and improves decision-making with up-to-date progress data

  1. {color:#000;}Predictable Costs and Schedule

–Fixed duration iterations improve predictability, easier to check the amount of work the development can perform in fixed-schedule time boxes

–The cost each learning objective (LO) is easier to see, giving more information that may be used in rapid trade-off decision making when prioritizing LOs.

  1. {color:#000;}Mitigates Risk and Improves Predictability

Less buyer risk when trying contractors/suppliers with which they have no prior experience working together

–Reduced risk of not getting the learning experience the buyer needs

–Less contractor/supplier misunderstanding of buyer requirements after first few iterations

–Prioritizes high-risk aspects of development, enabling early risk reduction

–Most challenging chunks of the learning experience prioritized first so the risk decreases towards the end of a project

  1. {color:#000;}Collaboration

–Buyer stakeholder engagement increased using iteration demos

–Smaller batches of work expected by the buyer’s subject matter experts (SMEs)

–Accommodates as much buyer involvement during design and development as desired

–More collaborative relationship with contractors/suppliers, and less adversarial because iteration demos allow transparency of progress

  1. {color:#000;}High Quality

–Earlier and improved buyer opportunities to give feedback to the development team

–More opportunities to check contractor/supplier responsiveness incorporating buyer feedback

–Improved quality discovering and resolving defects earlier

–Getting feedback about how well the team is meeting customer expectations earlier

–Leads to less rework as the project’s end-date draws near

  1. {color:#000;}Allows Change

–Easier to respond to changing buyer business conditions or stakeholder needs during the project with less impact (financial, schedule)

–Easier to negotiate for swap-out of similarly sized efforts even well into the project (for example, swap requirements E, F, G with requirements X, Y, Z instead)

–Less contractual quibbling because of the flexibility of the approach (with Lean-Agile contracts)

–Buyer determines the priority of LOs during the project, helping the development team to deliver the LOs that provide the most buyer value.

–Delivers the project needed at the end, not the one requested at the beginning

If they want the benefits of Lean-Agile when they buy any training that is not pre-built and pre-packaged, they will have to add a requirement that Lean-Agile be used in the development. Buyer groups will also have to carefully select which suppliers have sufficient expertise with Lean-Agile to meet the expectations of themselves and their stakeholders. They will have to adjust their reporting expectations of their chosen suppliers. They may want to join the supplier’s daily scrum/standup meetings. They will want to join all iteration demonstrations to see the working learning experiences made to date and inspect how well it meets their requirements and expectations. Later sections cover the changes buyers need to make in more detail.

[]5.10.2. A Training Group Supporting Software Product Development

Some groups support organizations that produce software products for enterprise use or for departments or small businesses. It is very likely that these groups have already heard of Lean-Agile and possible that the software teams they support already use Agile methods like Scrum to create the software. Adapt some of this book by documenting training for the product increment during each iteration as a member of a cross-functional team developing the software product. These teams also have to carefully time when to create training to avoid rework. If they attempt to build training too early in the product development lifecycle, before the design is stable they will experience much rework. Training professionals should be sure to join the daily scrums to keep in the loop and support the software product. Customers may come to your offices for training or join virtual and online training you develop to learn how to get the most out of the complex software product suite your organization makes. These teams will likely need to adopt the Lean-Agile terminology used by their parent organization. For example, if the software group uses Agile Scrum as a method, then rather than using a generic term like “iterations”, you will gain more traction if you use “sprints” and all the other Scrum terminology since your stakeholders already use those terms.

[]5.10.3. A Training Group Supporting Hardware Product Development or Manufacturing of Systems

Some groups support engineering groups that make hardware systems like weapon systems, semiconductor equipment manufacturing, and other manufacturers. These groups often produce training to operate and maintain the systems their employer builds and sells. They will likely have in-house SMEs to validate their training content. They may have training facilities with labs that include the equipment used during instructor-led training. These groups may already be familiar with Lean Manufacturing, but may not have considered using Lean for learning experience development yet. These groups can gain much from Lean-Agile. Building training for existing systems being manufactured is easier. Sometimes, these teams also have to support new system models and then must carefully time when to create the training to avoid excessive rework that inevitably occurs if they attempt to build training too early in the product development lifecycle, before the design is stable. Training professionals may have to use Lean-Agile by themselves if the larger organization is not using it yet. Another issue for these training groups is the overloading of terms. Lean Manufacturing means different things by Kanban than the Kanban board we discuss in this book. Be sure to compare and contrast Lean-Agile for learning experience development as we describe it against the literature on Lean Manufacturing so that you know when to clarify with your internal audiences.

[]5.10.4. An L&D Group Supporting a Human Resources, Learning and Development Organization or Sales Training

Some groups produce in-house training for their own organizations to help the organizational members better perform and help the organization meet its goals. In companies, this may include New Employee Training and Leadership Development curricula as part of a larger program of operational assignments and mentoring. Training professionals in these organizations may use Lean-Agile by themselves if the larger organization is not using it yet. Prioritization of what gets created first can sometimes be an issue for these training groups. These groups can also use Kanban boards to track the progress of any contracted suppliers creating custom learning experiences, games or simulations that are not pre-packaged or simply configured for your organization.

[]5.10.5. A Training Group Supporting a Services Company or Organization

Some groups create training to support the services that their organization provides to customers. Applying Lean-Agile in this environment may be more difficult, but it is still possible. Focus on the training scope first and gain successes.

[]5.10.6. A Training Group Selling Training Products to Businesses and Governments

Much of this book is written from our perspective of a training group that builds custom-made learning experience products for paying customers. This produces certain influences on how you might adapt Lean-Agile. In particular, since these groups are not supporting the main product, but are themselves the creators of the primary product, the way they apply Lean-Agile is as the main group, not as a supporting function.

In this book, we will use the word buyer or customer to mean your internal customers if you primarily work with other departments or groups, and to mean external customers if you primarily work with customers outside the normal boundaries of your organization.

[]5.10.7. Volunteers in an Open Source Project

Because volunteers are geographically spread across the globe, using a virtual Kanban board like Trello could help such a volunteer project. The use of cards for work items with accountability avatars or initials allows all involved to see who is doing what, and to coordinate their efforts without laborious meetings to talk through status when the data on status is visually available to all with access to the Kanban board. This can work for either training or technical publications. A Kanban board is also helpful when the time zones of those participating are not conducive to synchronous meetings.

[]6. Proprietary Versus Open

Let’s distinguish between what belongs to Raytheon as intellectual property and what is in the community of practitioners.

Lean-Agile principles are taught in universities after having been applied in automobile manufacturing and presented at public conferences for more than two decades and in software development for over a decade. Agile and Lean principles are not owned by our organization. The literature is full of excellent discourses on Agile principles and Lean product development principles.

“Obey the principles without being bound by them.”

— Bruce Lee

Principles change little, if at all. Methods from their application change continuously as do the circumstances of their use.

“But examine the matter from first principles.”

— The Meditations book 11 by Marcus Aurelius

Like Thomas Edison, we have stood upon the shoulders of giants:

  1. {color:#000;}Toyota, which discovered many of the Lean principles and applied them to automobile manufacturing.

  1. {color:#000;}Jeff Sutherland, the co-creator of Scrum
  1. {color:#000;}David Anderson, who discovered and applied the Kanban principles at Microsoft and other places
  1. {color:#000;}The signers of the Agile Manifesto, aimed at software developers
  1. {color:#000;}Edward Deming, a quality guru
  1. {color:#000;}Software practitioners who have been applying these ideas for over a decade and sharing their experiences at public conferences and on the Internet

We saw that what these pioneers described could work for learning experience product development just as well as it works for software product development.

We developed our methods, reducing them to practice. Our specific methods are proprietary. We spent much time in experimenting to find what works and what does not work. We used the scientific method to confirm for ourselves that Lean-Agile principles are valid.

To get learning experiences created quickly, you can use companies like ours (the fast way) or you can develop your own Lean-Agile methods (the slow way) for your organization and then apply them to make your own courseware. Both paths to Lean-Agile work. We say the principles of Lean-Agile can help your organization to the degree you apply the principles. In this book, we will not share our specific methods because we use them to improve our own business. However, we will share some of our experiences from the journey. Then, rather like industry conferences, we also hope to hear from you and learn from your experiences when you share your journey.

We have noticed some people in the software Agile community trying to claim their particular method is the only way to “do” Agile. Yet in our experience, correct principles applied to specific context often result in tailored methods being more effective for that specific place and time. So we are wary of people who claim that their method is the only true way. We can all derive methods from principles. Some methods like Scrum and Kanban seem to have been proven superior over competing ideas, but even then the idea of Lean-Agile is to adapt and adjust as necessary. The goal is not perfect alignment with someone else’s step-by-step methods or even dogma, but improvement of your own situation.

We have adjusted our methods to apply the principles in newer conditions. Some teams are mature. Other teams are newer. Some people have had exposure to Lean-Agile, others are completely new to it. Our customers get more comfortable as they see the risks being reduced with each iteration. All of these and hundreds of other contextual details change how we currently apply the principles of Lean-Agile. So this book is not a recipe book with our secret sauce or step-by-step instructions. Rather it is a story of how we have applied these principles in our specific circumstances that hopefully helps you better understand what we do and perhaps helps you apply the principles in your own unique circumstances. Even if competitors discover our current methods, they should vary their own implementation because their context is different and the principles likely suggest different methods. Our methods will change over time as our own conditions change and as we strive for perfection.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{background:rgba(0, 0, 0, 0);}.
p<{color:#000;}. Following anyone else’s methods without understanding the underlying principles will leave you exposed to additional risk.

The prudent way is for you to take the time and learn the principles and how they relate to one another so you can adjust appropriately. Or hire us. Or hire a consultant.

Anyone can learn these principles and apply them to their own unique circumstances. We gained our understanding gradually from much experimentation. The slower way can work for you too.

None of the principles presented herein are new, we have simply applied the principles to fit training circumstances, courseware rather than software. Our teams have created software too, but it generally has served the purposes of the courseware as a junior partner, so we sometimes lump software into the general term courseware as the product that is our focus in achieving specific learning outcomes.

We have condensed what we’ve learned from the software community discussions and a few things about how we’ve applied the same principles to organizational training learning experience product development. |

Your efforts to adopt Lean-Agile may be rejected at first, depending greatly on how you present the ideas, your current organizational context and the opportunities at the time.

One could argue that although business schools teach the principles of business, some businesses seem to apply and execute those principles more effectively than others in their context.

We want to thank Jeff Sutherland for his contributions to Agile Scrum as one of its creators. We wrote the first draft of this book just before seeing Jeff’s book, “Scrum, The Art of Doing Twice the Work in Half the Time” at an airport bookstore. Due to many Internet descriptions of Agile including software jargon and methods, some people may have a hard time getting past the jargon to envision a future state in other domains. However, Jeff did a great job of explaining Scrum to more generalized audiences in his book and we recommend it for most any type of project with two or more people involved.

One of our customers, the United States Army, provides a similar pattern. They teach the principles of warfare first and then train with various methods of applying the principle in various conditions. The methods continue to change often with the circumstances, but the principles continue to stand the test of time. Many armies continue to learn the principles of warfare, but not all apply them equally well. It is the continual improvement of the methods used and the quality of the leadership that ultimately wins in the chaos and complexity of armed conflicts. So it is with Agile principles and business uses of Lean-Agile as a project management approach for making training.

Teach your teams the principles, continually improve your current methods and lead Lean-Agile initiatives well. Also share at training industry conferences about your journey.

[]7. Focusing on Buyers of Training

Readers on teams that produce training materials may want to skip this chapter.

[]7.1. Traditional Sequentially Executed ADDIE (aka Waterfall Method)

First let’s compare the traditional sequentially executed Instructional Design (ID) framework, called ADDIE (analysis, design, development, implementation, evaluation) [18], with the Lean-Agile approach. There are some groups who may say, “But ADDIE doesn’t have to be sequentially executed. ADDIE can be iterative.” We agree.

In applying instructional models, there tends to also be a project management approach either wrapped around or integrated into the instructional model/approach.

Traditional project management approaches typically include the following activities:

  1. {color:#000;}Definition (goal, objectives, work breakdown structure (WBS), resource requirements)

  1. {color:#000;}Planning (roles, responsibilities, deliverable sequencing, resource scheduling, risk management)
  1. {color:#000;}Implementation (monitoring performance, adjusting, recovering)
  1. {color:#000;}Closeout and Evaluate (how we did, lessons learned)

When a traditional sequentially executed ADDIE value stream is used, the customer expects to experience interactions with the contractor or supplier in the following order:

  1. {color:#000;}Analysis

  1. {color:#000;}Design
  1. {color:#000;}Development
  1. {color:#000;}Implementation
  1. {color:#000;}Evaluation (if they choose to use evaluation, and to what Kirkpatrick level)

When wrapping traditional project management approaches around ADDIE, you may add some phases or stages to the expectations:

  1. {color:#000;}Project Definition and Planning

  1. {color:#000;}Project Implementation






  1. {color:#000;}Project Closeout

When these sequentially executed ADDIE phases or stages are put on a project Gantt chart for scheduling, it tends to look like a stair case descending to the right over time. If water were poured down these stairs, water would fall from analysis to design, to development, etc. Such sequentially execution is often called the waterfall method because of this visualization of water flowing down from one step or stage to the next after the first is completed.

Figure 6. Water Flows Down the Phase Stairs from High to Low (durations are examples only)

The ADDIE Model of instruction systems design seems to have been originally developed [18] in the 1970s by Florida State University for the United States military.[19] The Software Development Life Cycle (SDLC), nicknamed the ‘waterfall’ approach, was also adopted by the U.S. DoD after a misunderstood presentation in 1970 by Winston Royce.[20]

Both ADDIE and SDLC can both be described as waterfall when executed in sequential stages.

[]7.2. A Brief Comparison Between the Lean-Agile approach and ADDIE

The following table depicts a brief comparison between Lean-Agile and ADDIE.

Think of the following table like a continuum with each column at each end of the continuum. Sometimes a hybrid solution is necessary that combines elements of both.

Table 1. Comparison of traditional ADDIE and Lean-Agile

table{background:transparent;}. |=~{color:#000;background:rgba(0, 0, 0, 0);}. Aspect|=~{color:#000;background:rgba(0, 0, 0, 0);}. Traditional ADDIE|=~{color:#000;background:rgba(0, 0, 0, 0);}. Lean-Agile Approach| |<{color:#000;background:rgba(0, 0, 0, 0);}. Fixed|<{color:#000;background:rgba(0, 0, 0, 0);}. Scope (Requirements)|<{color:#000;background:rgba(0, 0, 0, 0);}. Cost and Time| |<{color:#000;background:rgba(0, 0, 0, 0);}. Estimated / Negotiated|<{color:#000;background:rgba(0, 0, 0, 0);}. Cost and Time|<{color:#000;background:rgba(0, 0, 0, 0);}. Scope (content and functionality)| |<{color:#000;background:rgba(0, 0, 0, 0);}. Emphasis|<{color:#000;background:rgba(0, 0, 0, 0);}. The entire course processed at once (large batch)|<{color:#000;background:rgba(0, 0, 0, 0);}. Small chunks of the course processed at once (small batch) as completed and working “courseware increments”| |<{color:#000;background:rgba(0, 0, 0, 0);}. Drivers|<{color:#000;background:rgba(0, 0, 0, 0);}. Schedule driven, i.e. Plan-driven; often deterministic instead of stochastic (i.e. Monte Carlo methods)|<{color:#000;background:rgba(0, 0, 0, 0);}. Business value driven: Prioritization of scope by highest training value every iteration; scope juggling| |<{color:#000;background:rgba(0, 0, 0, 0);}. Specifications|<{color:#000;background:rgba(0, 0, 0, 0);}. Up-front, detailed original requirements documentation|<{color:#000;background:rgba(0, 0, 0, 0);}. Requirements as the current courseware backlog item cards in current priority order| |<{color:#000;background:rgba(0, 0, 0, 0);}. Desired functionality: Interactivity|<{color:#000;background:rgba(0, 0, 0, 0);}. Each interaction type described in documents|<{color:#000;background:rgba(0, 0, 0, 0);}. Each interaction type described in user stories and prototyped in early iterations| |<{color:#000;background:rgba(0, 0, 0, 0);}. Design Constraints|<{color:#000;background:rgba(0, 0, 0, 0);}. Learning Objectives constrain content and functionality|<{color:#000;background:rgba(0, 0, 0, 0);}. Same, Learning Objectives constrain content and functionality| |<{color:#000;background:rgba(0, 0, 0, 0);}. User Experience (UX)|<{color:#000;background:rgba(0, 0, 0, 0);}. Yes, for courseware interface|<{color:#000;background:rgba(0, 0, 0, 0);}. Same| |<{color:#000;background:rgba(0, 0, 0, 0);}. Design changes|<{color:#000;background:rgba(0, 0, 0, 0);}. Minimized as risk mitigation|<{color:#000;background:rgba(0, 0, 0, 0);}. Emergent design okay if lower priority scope is swapped out to mitigate impact| |<{color:#000;background:rgba(0, 0, 0, 0);}. Approach|<{color:#000;background:rgba(0, 0, 0, 0);}. Phased and sequential|<{color:#000;background:rgba(0, 0, 0, 0);}. Iterative and incremental| |<{color:#000;background:rgba(0, 0, 0, 0);}. Up-front detailed plans|<{color:#000;background:rgba(0, 0, 0, 0);}. Entire project planned in detail|<{color:#000;background:rgba(0, 0, 0, 0);}. Short term work (first iteration or rolling wave) planned in detail| |<{color:#000;background:rgba(0, 0, 0, 0);}. Up-front minimal plans|<{color:#000;background:rgba(0, 0, 0, 0);}. Not applicable, all planned up front|<{color:#000;background:rgba(0, 0, 0, 0);}. Longer term work (future rolling waves)| |<{color:#000;background:rgba(0, 0, 0, 0);}. Proof of progress|<{background:rgba(0, 0, 0, 0);}.

  1. {color:#000;}Extensive documentation of desired courseware and process steps

  1. {color:#000;}Infrequent in process reviews of content and functionality |<{color:#000;background:rgba(0, 0, 0, 0);}. Working courseware increments every iteration (content & functionality)| |<{color:#000;background:rgba(0, 0, 0, 0);}. Completed deliveries|<{color:#000;background:rgba(0, 0, 0, 0);}. At the project end (unless in-process reviews used)|<{color:#000;background:rgba(0, 0, 0, 0);}. End of each iteration| |<{color:#000;background:rgba(0, 0, 0, 0);}. Size of work in process inventory|<{color:#000;background:rgba(0, 0, 0, 0);}. Large batch|<{color:#000;background:rgba(0, 0, 0, 0);}. Small batch| |<{color:#000;background:rgba(0, 0, 0, 0);}. Size of work items|<{color:#000;background:rgba(0, 0, 0, 0);}. Large (course or lessons)|<{color:#000;background:rgba(0, 0, 0, 0);}. Small (lessons or learning objectives)| |<{color:#000;background:rgba(0, 0, 0, 0);}. Amount of customer involvement|<{color:#000;background:rgba(0, 0, 0, 0);}. Distributed at the start and at the end of the project in large batches of time|<{background:rgba(0, 0, 0, 0);}.
    p<{color:#000;}. Same amount, but distributed throughout the project in smaller batches of time

(same amount just distributed differently) | |<{color:#000;background:rgba(0, 0, 0, 0);}. Frequency of customer feedback|<{color:#000;background:rgba(0, 0, 0, 0);}. Less|<{color:#000;background:rgba(0, 0, 0, 0);}. More| |<{color:#000;background:rgba(0, 0, 0, 0);}. Schedule orientation|<{color:#000;background:rgba(0, 0, 0, 0);}. Stages or phases (design, development, etc.)|<{color:#000;background:rgba(0, 0, 0, 0);}. Iterations| |<{color:#000;background:rgba(0, 0, 0, 0);}. Need for SMEs|<{color:#000;background:rgba(0, 0, 0, 0);}. During design and development|<{color:#000;background:rgba(0, 0, 0, 0);}. Same, during design and development| |<{color:#000;background:rgba(0, 0, 0, 0);}. Risk impact if SMEs unavailable when needed|<{color:#000;background:rgba(0, 0, 0, 0);}. Higher costs, Delivery delays|<{color:#000;background:rgba(0, 0, 0, 0);}. That work item moved/swapped to another iteration to minimize impact| |<{color:#000;background:rgba(0, 0, 0, 0);}. Response to change|<{color:#000;background:rgba(0, 0, 0, 0);}. Purposely slow and difficult process for scope additions|<{color:#000;background:rgba(0, 0, 0, 0);}. Purposely fast and easy process for scope swaps| |<{color:#000;background:rgba(0, 0, 0, 0);}. Continual Improvement|<{color:#000;background:rgba(0, 0, 0, 0);}. Lessons learned from previous project/program implemented on next project/program|<{color:#000;background:rgba(0, 0, 0, 0);}. Lessons learned from previous iteration implemented on next iteration| |<{color:#000;background:rgba(0, 0, 0, 0);}. Budget|<{color:#000;background:rgba(0, 0, 0, 0);}. Yes|<{color:#000;background:rgba(0, 0, 0, 0);}. Yes| |<{color:#000;background:rgba(0, 0, 0, 0);}. Stakeholder identification|<{color:#000;background:rgba(0, 0, 0, 0);}. Yes|<{color:#000;background:rgba(0, 0, 0, 0);}. Yes| |<{color:#000;background:rgba(0, 0, 0, 0);}. Role clarification|<{color:#000;background:rgba(0, 0, 0, 0);}. Yes|<{color:#000;background:rgba(0, 0, 0, 0);}. Yes| |<{color:#000;background:rgba(0, 0, 0, 0);}. Approvals as risk mitigation|<{color:#000;background:rgba(0, 0, 0, 0);}. Formal hierarchical review and approval before starting the next phase|<{background:rgba(0, 0, 0, 0);}.
p<{color:#000;}. Priority of work aligned with customer’s highest priority functionality and content (work items):

  1. {color:#000;}Product Owner prioritizes all work items in the courseware backlog to match customer’s priorities

  1. {color:#000;}Development Team chooses the highest priority backlog items to implement during each iteration

    Quality Assurance Inspections and Testing Quality efforts are tacked on at the end of the project Quality efforts are built-in, distributed across each iteration using the “definition of done” for every process step
    Sequencing of what Reviewers inspect and evaluate Courseware sequence order (the order in which the learner sees it) By priority (the most technically challenging and highest priority work items implemented first as a risk mitigation method).
    Evaluation (Formative) Progress check items are traceable to Learning Objectives[* Same*]
    Evaluation (Summative) Assessment test items are traceable to Learning Objectives[* Same*]
    Evaluation (Content & Functionality) Customer gives feedback at final delivery Customer gives feedback at each iteration, earlier in the courseware life cycle
    Evaluation (Acceptance) Customer evaluates reviewer comment incorporation and accepts entire course at final delivery Customer evaluates reviewer comment incorporation and accepts each iteration, which when all are accepted the course is accepted as the aggregation of the approved increments
    Risk impact if new scope added Higher costs, delivery delay Scope swap: New scope replaces an equal amount of lower priority work items. Highest priority work items delivered.
    Risk impact of requirements changes Higher costs, delivery delay Reprioritize work items. Highest priority scope gets delivered. Some lower priority work items dropped to meet cost/time goals. Smaller batch size reduces WIP inventory minimizing change impact.
    Risk impact if effort is harder than planned Higher costs, delivery delay Highest priority work items delivered, lowest priority work items dropped to meet cost/time

[]7.3. Determining How Much Lean-Agile a Buyer Organization can Apply

When we want to optimize project/program outcomes to deliver more value, improve quality, shorten lead time and reduce costs, then incorporating as much Lean-Agile as we can will help us to achieve those improvement goals.

Much of this book has described either Waterfall or Lean-Agile as if the two poles on opposite ends of a continuum are the only two choices: Either full Lean-Agile or full waterfall as shown in the following figure.

Figure 7. Waterfall Versus Lean-Agile

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Lean-Agile does not require the entire enterprise to use an Lean-Agile approach to provide value.|

[]7.3.1. Obstacles to Full Lean-Agile Adoption

Yet your specific circumstances may not yet fully support a 100% Lean-Agile approach.

Because the training industry is still new to Lean-Agile, you may experience obstacles that prevent full adoption of Lean-Agile in one large bite or batch. These obstacles may include:

Obstacles to Lean-Agile

  1. {color:#000;}New ideas are avoided or killed off rather than explored and judged for value potential
  1. {color:#000;}Blame culture where any mistake is punished, no underwriting of failure that progresses towards improvement
  1. {color:#000;}Policy sclerosis – Existing governance policies which allow little to no options for changes
  1. {color:#000;}Low trust exists between buyer and contractors/suppliers
  1. {color:#000;}Interactions between buyer and contractors/suppliers are unsupportive of win-win outcomes
  1. {color:#000;}Legal agreements are designed to support waterfall methodology
  1. {color:#000;}Lack of legal support to craft Lean-Agile contract provisions
  1. {color:#000;}Plans are almost all deterministic, very few probabilistic, indicating low current level of planning for uncertainty
  1. {color:#000;}Plans that are treated as infallible and conformance to the plan is the only acceptable way
  1. {color:#000;}Being unwilling to deal with things as they are, but only as you want them to be [21]
  1. {color:#000;}Persistent reversion to established mental frames (ways of doing and ways of seeing within the organization) [22]
  1. {color:#000;}Systematic withholding or non-circulation of information (organizational secrecy) [22]
  1. {color:#000;}Business process inertia
  1. {color:#000;}Leadership skills of those involved (tendency of leadership through dominance)
  1. {color:#000;}Stakeholders with knowledge of particular Agile methods for software (a) who expect exact copying from software methods, and (b) who object to principle-based tailoring of methods for a courseware project
  1. {color:#000;}Organizational culture resembles fiefdoms of the middle ages with turf-protection behaviors trumping collaboration
  1. {color:#000;}Low tolerance for a high degree of transparency
  1. {color:#000;}Messengers get shot when delivering bad news
  1. {color:#000;}Contracts used as flogs in antagonistic beatings between buyer and producer
  1. {color:#000;}Estimates made during planning quickly harden into the absolute reality that must occur
  1. {color:#000;}The idea that team members are replaceable parts
  1. {color:#000;}Complexity of projects is low (simple projects can still work with waterfall/traditional approaches)

[]7.3.2. Factors that Support Complete Lean-Agile Adoption

On the other hand, the following may help you move to a more full implementation of Lean-Agile:

Facilitators to Lean-Agile

  1. {color:#000;}Customer-focus and customer expectation of faster speed
  1. {color:#000;}Innovation friendly culture
  1. {color:#000;}Complex projects
  1. {color:#000;}Continuous improvement that is practiced, not just preached
  1. {color:#000;}The idea that failure precedes improvement
  1. {color:#000;}Being willing to deal with things as they are, not as you want them to be [21]
  1. {color:#000;}Lean-Agile contracts

[]7.3.3. Hybrid Lean-Agile

You may have to apply the principles to your specific situation and tailor a hybrid approach as an initial iteration towards a larger Lean-Agile change over a longer time period.

Figure 8. Hybrid Approaches Between Waterfall and Lean-Agile

If you discover obstacles, you can try to overcome any obstacles first and then go full tilt Lean-Agile. A more practical approach may be to work with the contractor/supplier to design a hybrid approach that fits your circumstances better for now and use continual improvement to adjust over time.

[]7.4. How the Lean-Agile Approach Impacts the Buyer’s Risk

The following table describes what changes buyers can expect when Lean-Agile is implemented into the process and when these changes will have their greatest impact.

Table 2. How Buyer’s Risks Change with Lean-Agile

Traditional ADDIE|=~{color:#000;background:rgba(0, 0, 0, 0);}. Lean-Agile Approach
Risk increases towards the end of the project with waterfall. Risk decreases with each iteration in Lean-Agile.
Because the extensive documentation is the measure of progress, hidden misunderstandings within textual meaning can delay substantial changes until late in the development lifecycle, increasing costs and delaying the schedule due to the rework needed to recover. Because the actual working courseware is the measure of progress, there is less chance for misunderstandings from individual interpretation of textual requirements, decreasing the rework and increasing the likelihood that the course will be delivered on time.
The most challenging work items may not get addressed until late in the project, making it difficult to modify the project to recover from a problem. The most challenging work items can be prioritized highest so they are addressed as early in the development lifecycle as possible.
The buyer cannot see the contractor’s/supplier’s large inventory of work items that are work in process (WIP) and partially complete or incomplete. The contractor/supplier has so much money tied up in this WIP inventory that they may have a financial incentive to argue and oppose all changes as the buyer’s situation may change. The buyer can see the entire increment’s worth of courseware in process. The contractor/supplier has minimal WIP inventory, so changes can be negotiated for swap-out without as much rework to partially complete work items, and with less financial burden, easing the discussion and process around changes when they are necessary. This also helps when priorities or constraints become more fluid.
It is difficult for the buyer to assess progress. It is easy for the buyer to assess progress by inspecting each iteration’s increment that includes only working courseware.
It is difficult for the buyer to assess how closely the solution matches the problem. It is easy for the buyer to assess how closely the solution matches the problem because each increment works.
The buyer’s expectations are encoded into text in extensive documentation, requiring decoding of those expectations by the producer, driving a hidden risk of missed value too late in the development life cycle. The buyer’s expectations become even clearer after the first iteration demo conversations as the buyer inspects and evaluates one increment (regardless of how well or poorly the documentation described what they really wanted). The buyer’s expectations are refined and communicated to the development team with each delivered iteration, reducing rework and the risk of not providing the desired value.
The risk of contractor/supplier staff turnover is larger to the buyer if the person who worked on a particular part of the courseware may no longer work on the team when feedback finally identifies an issue or defect, potentially months later. This delay potentially leaves a new person trying to glean the original developer’s intent when addressing a problem. The risk of contractor/supplier staff turnover is smaller to the buyer because the staff that worked that increment two weeks ago are more likely to still be on the project and available to fix any defects discovered, ensuring the continuity of buyer feedback all the way through resolution as the defect is fixed in the next iteration two weeks later.
The buyer staff’s scheduling windows for review require large blocks of the reviewer’s time because the batch size is so large (entire course). The buyer staff’s scheduling windows for review can be much shorter because the batch size is much smaller, taking less of their time to review.
Stakeholders have such a large batch to review that they tend to review asynchronously and alone for efficiency, increasing the chances of contradictory feedback that requires another round of tracking and review to referee the conflicts. All stakeholders are encouraged to attend each synchronous, end-of-iteration demonstration to provide feedback to the development team, with other buyer staff listening, which helps to quickly solve any mutually exclusive feedback.
The risks increase because specialists are working alone for long periods and then simply handing off to the next specialist. The risks reduce because all needed specialists are on a cross-functional teams see each other’s inputs more frequently, catching problems before as much time has passed.
The risks are often hidden with non-visible knowledge work items and consequently not addressed until much later in the life cycle when the problem is finally noticed. Day-to-day risks are managed at the daily cross-functional team stand up meetings in front of the visual board, making blockers visible, bottlenecks visible, and the state of all the components obvious.
Changes late in the lifecycle increase the risk of the project costing more than budgeted. If Agile contracts were used, and the buyer leadership gets that Agile scope (requirements) is flexible, then changes can be swapped out for equally sized effort even later in the lifecycle.
Waterfall tends to fix the scope of the requirements. This implies that all work items are of equal priority. This can lead to implementing critical items late, when project managers assume they will all be implemented, so order is less important. Sometimes, this may eliminate the option of on-time delivery if all does not go well with a critical item. This, in turn, leads to delays in schedule and sometimes an increase in costs. With the development team always pulling the next highest priority work item, the buyer is assured of getting their highest priority requirements and features within the flexible scope, within the fixed time and fixed cost.
Higher-level risks are addressed at the program level, following the program risk management process. Same, no change
Political risk for buyer’s staff inside their own organization with their stakeholders if the project is not successful. Same, no change

So in summary, the biggest changes in buyer’s risk are:


  1. {color:#000;}Lean-Agile decreases risk with each iteration.
  1. {color:#000;}Lean-Agile demos reduce misunderstandings with every iteration.
  1. {color:#000;}Lean-Agile can improve contractor/supplier flexibility.
  1. {color:#000;}Easier for the buyer to assess actual courseware progress.

[]7.5. How Agile Changes the Buyer’s Training Procurement Process

The change from large-batch to small-batch processing is the biggest change to the buyer’s processes.

  1. {color:#000;}The course design document may not change much.

  1. {color:#000;}Rather than see a storyboard for the entire course (large-batch) they will see only the storyboard for the increment or smaller portion of the courseware.
  1. {color:#000;}Rather than all the portions of the course stack up, waiting until the entire course is on the storyboard and approved before any development happens, a smaller batch of the course flows through storyboard and begins development with the rest of the course waiting until the contractor/supplier completes this smaller batch.
  1. {color:#000;}The increment the buyer inspected from iteration 1 gets their comments incorporated during iteration 2, so increment 1 can be done much earlier in the development lifecycle than with the waterfall approach. In waterfall ADDIE, the comments are not provided nor incorporated until nearly the end of the lifecycle.
  1. {color:#000;}Rather than only having stakeholders and subject matter experts review the courseware in a self-paced, asynchronous way, alone, they can also join the key conversations in a group-paced, synchronous way, together during iteration demos. Although timebox limits exist, these demos rarely demonstrate everything, but they do aim to evoke the most useful conversation possible between buyer’s staff and producer’s staff. Those stakeholders and subject matter experts concerned about more work are also protected because the amount of content in an increment is only what the contractor/supplier can build in a two-week period, the amount of content to review is substantially less than the entire course in one large batch of reviewer effort.
  1. {color:#000;}As increments get completed, the courseware is built up increment ‘layer’ by increment ‘layer’ into its whole when the last increment is completed.
  1. {color:#000;}Pilot testing of the courseware can be done in one large batch as before, or the buyer can now conduct the pilot course one increment at a time (especially for distance learning or if no travel is involved for pilot participants), parallel with development of the next increment. This can speed incorporation of the pilot participant’s feedback. Applying an iterative process to pilots is optional.
  1. {color:#000;}For buyers that desire lessons learned sessions with their contractor/supplier, the collection of lessons learned feedback should occur every two weeks, and tends to be mostly complete by the time a waterfall lessons learned session begins. This closer-to-the-action collection of lessons learned data tends to provide qualitatively better lessons because participants don’t have to search their memory from months ago, but for only the last two weeks.

[]7.6. What the Buyer Should Tell their Staff to Expect from Lean-Agile

The biggest change for buyer’s staff is moving from large batch to small batch. Lean’s small-batch processing means we only work on a few items at a time, completing one, and then moving on to the next work item. This changes the experience of all involved. Anyone formerly used to large-batch processing has most of their expectations altered by this small-batch processing approach. The following figure shows the change in approach.

Figure 9. Lean-Agile versus Traditional ADDIE, From Horizontal Slices to Vertical Slices

The easiest way of describing this is large-batch versus small-batch. Traditional ADDIE is often performed in large batches. The following figure depicts all the work for analysis performed on the entire course before moving to design. The entire large batch gets processed before anything moves on.

Figure 10. Large-Batch Movie – Scene 1

The next scene in the large-batch movie of progress shows the entire course moving to the next phase. Again, all the design work for analysis is performed on the entire course before moving anything to development. More of the work waits on the rest before proceeding through the workflow.

Figure 11. Large-Batch Movie – Scene 2 – Entire Large Batch Processed at each Step

As the large-batch movie continues, yet again, all the development work gets performed on the entire course before moving anything to implementation.

Figure 12. Large-Batch Movie – Scene 3 – Entire Large Batch Processed at each Step

Okay, that’s enough of that movie to see how it plays out. Now let’s switch to small batch processing.

Contrast the previous large-batch movie of progress with the following small-batch movie of progress. Scene one shows the team working on smaller chunks of the course. Smaller batches of work effort move along the entire workflow without waiting until the entire course is processed for that workflow step.

Figure 13. Small-Batch Movie – Scene 1

As we continue to watch the small-batch movie, scene two shows the team continuing to work on smaller chunks of the course. Note that the improvement ideas they come up with (illustrated by the dolly) do not have to wait until the entire course is processed to be applied. These improvements tend to speed the workflow.

Figure 14. Small-Batch Movie – Scene 2

The preceding series of batch movies shows the big picture.

Next, let’s drill down to another level of detail. Rather than just simple blue boxes, now we’ll depict training-specific work items. The following figure shows the work effort for each ADDIE part of the workflow represented as A for analysis, LO for learning objective, Eval for evaluation (formative or summative), LL for lessons learned, and Pilot for validating the courseware with sample students, etc. Note that the granularity of these work items is intentionally lower than lessons, and reduces in size to learning objectives (LOs) instead. This is because LOs tend to vary in size of effort less than lessons tend vary in size of effort.

Figure 15. The entire work effort for each ADDIE part of the workflow

Just as the preceding large batch movie, the following large batch movie performs all work items in one large batch for each step in the ADDIE workflow.

Figure 16. Large-Batch Movie – Scene 1 – Detailed Work Items

Figure 17. Large-Batch Movie – Scene 2 – Detailed Work Items

Figure 18. Large-Batch Movie – Scene 3 – Detailed Work Items

Figure 19. Large-Batch Movie – Scene 4 – Detailed Work Items

Figure 20. Large-Batch Movie – Scene 5 – Detailed Work Items

Figure 21. Large-Batch Movie – Scene 6 – Detailed Work Items

Figure 22. Large-Batch Movie – Scene 7 – Detailed Work Items

Now for the Lean-Agile mind shift. Let’s switch to small-batch processing, and have the following small-batch movie showing the detailed work items too. The entire courseware effort has been split into ten iterations. This small-batch movie will only show a single iteration. Also because we’ll cover Lean-Agile planning in later sections, we’ll start this movie from Analysis.

Figure 23. Small-Batch Movie – Scene 1

Figure 24. Small-Batch Movie – Scene 2

Figure 25. Small-Batch Movie – Scene 3

When this movie ends for iteration one, the iteration’s scope is complete. Note that only ten LOs are included in the increment’s scope, in this example, rather than the entire course worth of LOs. Each LO shows some design and development effort. The lessons learned are distributed across the iterations too, allowing the team to make improvements sooner.

Figure 26. Small-Batch Movie – Iteration 1 Done

When this movie ends for iteration two, the following scope is complete with both iteration 1 and 2 work done.

Figure 27. Small-Batch Movie – Iterations 1 and 2 Done

Adding one iteration batch at a time continues until the entire course is done. When all the Agile iterations are done, the course is completed. The following figure shows how each small batch was completed like a layer cake.

Figure 28. All Iterations – Using Smaller Batches to Build One Increment at a Time

Notice that instead of the buyer stakeholders reviewing the entire course analysis, and then much later reviewing the entire course design, and then later reviewing the entire course development, Lean-Agile changes the approach. They buyer’s stakeholders using Agile will interact with the development team more frequently because each iteration is only two weeks in duration. The buyer’s stakeholders do NOT need to spend more time reviewing the courseware increments when using Agile. Instead, they simply split the large batch of review effort into smaller batches of review effort and perform those smaller efforts more frequently.

Some organizations stakeholders initially seem to prefer to focus on scheduling their stakeholders efforts, like they were traditionally used to doing. Then we help them realize that the traditional scheduling of a large chunk of time was a countermeasure to address the difficulty of finding such large batches of review time. Large batches of review were so disruptive and hard to find that they compensated by scheduling it as far in advance as possible.

To help change the mindset, we sometimes coach stakeholders about why their more frequent collaboration is so vital at each iteration, and how the feedback provided improves quality and reduces the buyer organization’s risk by inspecting what they expect every two weeks rather than waiting longer and potentially having more hidden issues get discovered later in the project lifecycle.

It has proven helpful to remind all involved that the earlier we can fix expectation mismatches or defects, the less risk exists to both schedule and budget. When they see that each iteration only requires a 1–2 hours of their time rather than 10–20 hours, then they are willing to try it. Because the increment size is constrained by what can be built in two weeks, it tends to be more manageable in a short iteration demo/review.

  1. {color:#000;}Let the buyer’s reviewers know that the iteration demo is the main place for feedback to the development team and for review of key areas.

  1. {color:#000;}Let the buyer’s staff know that the cadence is faster than with traditional projects.

Though the demo does not prevent personal asynchronous reviews outside the demo, it is critical that all reviewers stay in-flow with the short cycle times.

The task to referee potentially conflicting feedback quickly still applies with Lean-Agile.

The buyer’s people are needed more frequently, but for smaller durations. In the end, the amount of time each person spends reviewing should be the same as a traditional project, but it is distributed across each iteration rather than batched all at the end of the development lifecycle.

  1. {color:#000;}Let buyer SMEs know that our instructional/LX designers may seek feedback more frequently.

Our instructional/LX designers may request input with a short turnaround expectation, but the scope is small, so it should not interfere too much with their regular duties. The benefit to buyer SMEs is that rather than having to switch their attention back and forth amongst work items in the entire course, they only have to focus on the courseware increment based on the committed iteration goal for that two weeks. This one-piece flow helps reduce what Lean calls ‘context switching waste’ when people have to wrap their head around something they haven’t touched in a while. This prevents our requests for SME support from wiping out the SME’s entire day. Supporting only a small work item per day means the calls with us are typically short. SME timely responsiveness is still crucial to maintain the fast cadence both parties are using.

  1. {color:#000;}Notify the SMEs of key input period.

It is important for the buyer’s SMEs to note the timing of the courseware increments on the schedule because this is where the majority of their inputs are required. SMEs may also want to review the backlog to see how the LOs they need to support are prioritized. This helps the buyer’s SMEs allocate their schedules better.

  1. {color:#000;}The iteration retrospective meeting is an integral part of the Plan-Do-Check-Adjust (PDCA) process.

So if buyer staff want to bring up an improvement, the demo is the place for it. They should not wait until the end of the course development lifecycle to mention their change or improvement idea.

  1. {color:#000;}Because the buyer will have many chances to redirect priorities, be cautious of using an antagonistic stance between the buyer and the contractor/supplier.

Lean-Agile includes a principle of respecting people. Lean-Agile aims at frequent collaboration between the buyer and the contractor/supplier. This collaboration is aided by a friendly tone of communications between both parties. Because you will know more surely the state of the progress during the lifecycle, it is less necessary to start off using the contract as a bludgeon, or behaving as if all contractors/suppliers are out to take advantage of you. Use the data from each iteration demo to determine how to respond to the contractor/supplier.

  1. {color:#000;}Be sure your stakeholders know the inspect and adapt or PDCA cycle of Lean-Agile is intentional.

Every engagement between the buyer and contractor/supplier includes some degree of getting to understand each other and what is meant by specific statements and behaviors. Even the best thermostats overshoot as they transition to the target. If buyer stakeholders want ‘zero defects’ or think of adaptations as failures, and they interpret those adaptive efforts as bad, then the buyer will have uncomfortable meetings. It is better to set that expectation up front. All endeavors with humans have some degree of adaptation. Lean-Agile has a systemic way of moving both the contractor/supplier development team(s) and the buyer’s staff and stakeholders towards improvement. Lean calls it ‘strive for perfection’, ISO 9001 calls it continual improvement. Agile has mechanisms to ensure these improvements are getting incorporated into your project. Conduct a communications campaign with all stakeholders during kickoff meetings to help this be a common understanding among all parties. Help stakeholders notice the accretion of adaptive improvements over the project lifecycle and help them interpret that as a good thing.

[]7.7. How Training Buyers Can Determine Who Really Knows Lean-Agile

To really know who to trust to take your important training needs down the Lean-Agile path, the buyer will have to determine which contractor/supplier to select. The following may help make the best selection.

  1. {color:#000;}Ensure the contractor/supplier knows the principles of Lean-Agile, and that they are not just mimicking someone else’s Agile method such as Scrum or Kanban. If using request for proposals (RFPs), be sure to leave sufficient page count in your maximum page constraint and require they explain how their methods tie back to the principles of Agile and Lean. This will allow the contractor/supplier to adjust to the specifics of your situation rather than increasing your risk by blindly following someone else’s methods without the proven capability to adapt.

  1. {color:#000;}Ask how the contractor/supplier has used Lean-Agile before and what they learned and how they adapted during that experience.
  1. {color:#000;}Be absolutely sure that the contractor/supplier has experience applying Lean-Agile to the training domain and not only the software development domain. There are significant differences.
  1. {color:#000;}Be sure to find out how they apply the Theory of Constraints (TOC) because it guides the application of Lean exceedingly well. If the contractor/supplier uses TOC to constrain their Lean efforts, you are more likely to achieve your cost and schedule targets than if they use Lean unconstrained.
  1. {color:#000;}Also, ask how much experience the contractor’s/supplier’s Lean-Agile Coach has. Their coaching of the contractor/supplier cross functional teams will increase the likelihood of success.
  1. {color:#000;}If using RFPs, be sure to describe your organization’s degree of acceptance of Lean-Agile approaches and ask the contractor/supplier to clearly describe how much Lean-Agile they think is appropriate to your specific circumstances so that the hybrid solution is fit-to-purpose and stands the most chance of success within your organization and with your stakeholders. From their rationale about why they will adapt this or that, you will gain insight into how well they really know Lean-Agile. Those that simply mimic someone else’s method have difficulty synthesizing alternate solutions.
  1. {color:#000;}The use of martial arts belts as analogies has taken an unfortunate turn in Six Sigma and Lean because many people assume that if some third-party has provided black belt or green belt status that the contractor/supplier is good to go. Rather than looking for some third party black belt in Lean as your primary or only selection criteria, be sure you can distinguish whether the contractor/supplier can perform Lean-Agile like a martial arts black belt or if they simply market well but perform as a white or yellow belt. Because we’re training professionals, we’ll use Bloom’s Taxonomy to reinforce this point. Select a contractor/supplier that can perform Lean-Agile at Bloom’s “creating” level rather than simply the “applying” level.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. We suggest reading Velocity: Combining Lean, Six Sigma and the Theory of Constraints to Achieve Breakthrough Performance – A Business Novel, by Dee Jacob, Suzan Bergland, and Jeff Cox for an example of Lean carried out by a well-meaning but inexperienced practitioner where the sponsor paid a difficult price for their on-the-job learning. It is based on a manufacturing scenario, but the principles of how to use TOC to constrain Lean are well illustrated in the story.|

[]7.8. How Much More Time will Lean-Agile Take for Buyer’s Staff

Lean-Agile simply takes the big chunk of time that the Buyer’s staff already uses in traditional ADDIE waterfall and spreads the efforts out, distributed across the iterations on the schedule.

Therefore if implemented well, there should be no extra time spent. See the following caution.

|={color:#000;background:rgba(0, 0, 0, 0);}. Caution|<{color:#000;background:rgba(0, 0, 0, 0);}. Where the buyer staff can increase their time spent is if they try to formalize and add extensive documentation requirements to every iteration event. For example, if the buyer staff simply has written minutes from each iteration demo, that should be sufficient for following up to ensure the contractor/supplier incorporates all of their feedback. Resist the habits of extensive documentation as the proof of progress and focus on working courseware as the proof of progress.|

[]7.9. Behold Complete Transparency

Buyer stakeholders tend to want to get a reliable sense of how their work is progressing.

It is easier in a manufacturing environment because manufacturing floors have physical widget assemblies and components that anyone walking by can see. The manufacturing workplace is visual by default. Buyer stakeholders may visit a contractor’s factory see for themselves the status of the work in process (WIP) inventory at a glance.

However, unlike physical widgets, knowledge work is invisible.

The lack of visibility of knowledge work makes tracking its progress a little harder. Traditionally, project/program managers have tried to use status reports of varying formats to get a sense of how the invisible WIP is progressing. Many of these reports leave hidden issues that increase risk.

Lean asks us to make even knowledge work visible for all. So to make it visible to both the development team and the buyer stakeholders, let’s use an Agile Kanban board to make the WIP inventory visible. Say we use a white board and sticky notes for work items.

One of the biggest potential changes for buyer stakeholders is getting invited to the contractor’s/supplier’s Agile Kanban board. Some practitioners call them information radiators. Some call them Scrum boards. For the sake of consistency, in this book we’ll call them Kanban boards.

Now some project/program managers don’t like the idea of total transparency of our work in process. They only want to focus on the traditional monthly or quarterly in-process reviews instead. Data from the Kanban board can still be summarized for these longer term reports if desired. Yet the biggest advantage is that any buyer stakeholder can visually see the daily status of the work if they want to.

We include a later section that describes Kaizen,[23] a Japanese word for continual improvement. The development team needs the daily view for daily Kaizen while seeking perfection (Lean).

So if the buyer stakeholders use the transparency, this is what they will see.

First, let’s monitor a traditional waterfall (large batch) project on our visual Kanban board. We start with the backlog plan, what engineers often call the requirements stack.

Figure 29. Start Point – Course Batch Size

Then we move the entire batch in to WIP.

Figure 30. Traditional ADDIE Work In Process – Course Batch Size

Then we wait many days, or weeks or even months to see any movement.

Figure 31. Traditional ADDIE Done – Course Batch Size

Notice that with traditional, large batch ADDIE process phases the large batch size does not provide much information about how things are going.

So let’s compare that to the Kanban board in a small batch Lean-Agile approach instead. We start with the backlog plan, what engineers often call the requirements stack.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Because this is a dynamic process, it takes a series of images to see the “movie”, or to depict the motion across the Kanban board over time.|

Figure 32. Increment 1 WIP – Increment Batch Size

Figure 33. Increment 1 Done Except for Rework

Figure 34. Increment 2 WIP

Figure 35. Increment 1 Rework is Done

Figure 36. Increment 2 Done Except for Rework, Increment 3 WIP

Figure 37. Increment 2 Rework is Done, Increment 3 is Done Except for Rework

Figure 38. Increment 4 WIP, Rework for 3 still WIP

And so on and so on. These figures were only an example. Actual work depends on many factors.

With the smaller batch view, much more information is available than with the course batch size example. You can go to smaller batch sizes to improve flow.

[]7.10. Lean-Agile and Earned Value Compatibility

Many commercial companies have no requirement to use Earned Value and may want to skip this section completely.

EVM was created when sequential, or waterfall project management approaches dominated. Lean-Agile provides a project/program management approach to propose solutions, execute projects, and produce products for our customers. Are the two compatible? Can Lean-Agile work when Earned Value is required?

[]7.10.1. A Brief Introduction to Earned Value Management

Contract work for the U.S. Government typically requires the use of Earned Value techniques.

If you’re not familiar with it, Earned Value Management (EVM) is a practice for program management that is used across the U.S. Department of Defense (DoD), the U.S. Federal government, and the commercial sector. Government and industry program managers use EVM as a program management tool to provide joint situational awareness of program status and to assess the cost, schedule, and technical performance of programs for proactive adjustment as needed.[24]

Earned Value is intended to increase the value to all stakeholders by using means for managing programs. The American National Standards Institute/Electronic Industries Alliance (ANSI/EIA) Standard EIA-748, Earned Value Management Systems, defines the requirements of an acceptable EVMS if you need more detail.

These guidelines consist of five categories: (1) organization; (2) planning, scheduling, and budgeting; (3) accounting considerations; (4) analysis and management reports; and (5) revisions and data maintenance.[25]

[]7.10.2. Recent Thinking Indicates Lean-Agile and EVM can Work Together

Recent activities related to U.S. Government contracting have focused on how to use Lean-Agile even when EVM is required.

In 2015, the Office of Performance Assessments and Root Cause Analyses (PARCA) [26] held an open meeting to discuss the nexus of Earned Value Management (EVM) and Agile development within a DoD environment.[27]

This meeting produced many insights that indicate quite a few organizations are successfully making Lean-Agile and EVM work together. The following is a list of some of the relevant insights:

  1. {color:#000;}Deliver timely, relevant solutions thru iterative and incremental delivery [28]

  1. {color:#000;}EVM is needed to drive efficiency [28]

–Demonstrates efficiency and provides input to needed course corrections

  1. {color:#000;}Some needs require continuous changes and upgrades across the lifecycle [28]

  1. {color:#000;}EVM is needed to drive consistent-objective results [28]

–Layers of incentives tend to drive overly optimistic promises of results [28]

  1. {color:#000;}Agile is a mainstream process used across commercial industries [28]

–Highly collaborative with consistent results

  1. {color:#000;}Impact to Core DoD Processes [28]

–Requirements: Move from a fixed set of requirements to evolving requirements and user’s role throughout.

–Delivery: Move from the static waterfall model to the Agile model with user feedback driving priorities

– Governance: Move from driven by milestones and breaches to more frequent review- delivery focused

–Functional Areas: Move from rigor tied to documentation for single milestone to rigor tied to demonstrated risk and delivery of capabilities

  1. {color:#000;}Changed the oversight and governance by replacing “trip wire” oversight to more frequent, less formal involvement [28]

[]7.10.3. Example to Demonstrate Lean-Agile and EVM Aligning

We agree with the broader community that Lean-Agile projects can be planned and executed to align with EVM.

Planning in Lean-Agile is distributed rather than all up front. Agile projects plan and execute in accordance with EVM. Agile promotes a strong, continuous planning discipline with the right level of precision for each planning time-horizon under consideration. Product Planning establishes the initial product backlog and product roadmap and provides the details needed to complete the initial schedule baseline and establish the performance measurement baseline (PMB).[29] In the following table, planning precision increases from the top row to the bottom row.

Agile Planning levels related to Agile EVMS [29]

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. The following sets of bullet lists were originally designed as a wide table in a layout oriented version of this book, but it was changed to a set of bullet lists to accommodate some eReaders that are portrait-only orientation. Newer devices can change orientation.|


Product Planning

Release Planning

Iteration Planning

Daily Planning



Product Planning

Project startup; updates throughout the project

Release Planning

Each incremental release/rolling wave

Iteration Planning

Each iteration

Daily Planning




Product Planning

Project Duration

Release Planning

Each incremental release/rolling wave

Iteration Planning

Each iteration

Daily Planning

1 Day



Product Planning


Release Planning

Features/ Stories

Iteration Planning

Stories/ Tasks

Daily Planning



Product Planning

Product Backlog; Product Roadmap

Release Planning

Release Plan

Iteration Planning

Iteration Backlog

Daily Planning

Updated Iteration Backlog

Some of these discussions meant ‘system’ to mean software applications, but we mean it as courseware for this book.

Sometimes a worked example helps to visualize what we mean. The following is an example [29] of planning with Agile and EVM.

Table 4. Sample Schedule for Agile Planning with Inchstones

EV Level|=~{color:#000;background:rgba(0, 0, 0, 0);}. Task Name|=~{color:#000;background:rgba(0, 0, 0, 0);}. Start|=~{color:#000;background:rgba(0, 0, 0, 0);}. Finish
CA WBS control account ABC Tue 5/18/17 Wed 4/2/18
WP WBS work package DEF Tue 5/18/17 Fri 11/8/17
milestone ——WBS milestone GHI Tue 5/18/17 Fri 7/19/17
milestone ——WBS milestone JKL Tue 7/22/17 Fri 9/13/17
milestone ——WBS milestone MNO Tue 9/16/17 Fri 11/8/17

With this schedule, the following inchstones are associated.

Table 5. Sample Inchstones List for Agile Planning with Inchstones [29]

Inchstone|=~{color:#000;background:rgba(0, 0, 0, 0);}. Weight|=~{color:#000;background:rgba(0, 0, 0, 0);}. Status|=~{color:#000;background:rgba(0, 0, 0, 0);}. Earned
GHI 39 started 31
User Story 1 3 done 3
User Story 2 3 done 3
User Story 3 8 done 8
User Story 4 8 done 8
User Story 5 2 done 2
User Story 6 3 done 3
User Story 7 2 not started 0
User Story 8 8 started 4
User Story 9 2 not started 0

When using Lean-Agile, handling inchstones correctly is important for EVM.

Supporting Data for Milestones (not subject to BCR) [29]

  1. {color:#000;}Stories are used as inchstones for earned value reporting to reinforce value of working [courseware] over completion of development tasks.
  1. {color:#000;}Inchstones are not IMS tasks and therefore do not require budget, scope or schedule assigned to them.
  1. {color:#000;}Story point weightings are used to calculate the earned value contribution of each completed story.
  1. {color:#000;}Total story points earned is used to calculate milestone percent complete used for earned value reporting.
  1. {color:#000;}Incomplete stories are planned for a subsequent iteration but remain with their parent IMS milestone.
  1. {color:#000;}If a started story implements scope outside the WP, you need to open a new WP.
  1. {color:#000;}Inchstones may be changed without a BCR as long as:

–Added stories are related to the features that comprise the scope of the associated milestone.

–Removed stories are not necessary to complete the scope of the associated milestone.

Management will still want to know where we’re at during the project, so a burndown or burnup chart can help show progress against the backlog.

Agile Execution: Release Burndown [29]

  1. {color:#000;}The Release Burndown shows the number of story points remaining at the end of each iteration.
  1. {color:#000;}The Release Burndown is a leading indicator of schedule performance.
  1. {color:#000;}In the example, Sample Schedule for Agile Planning with Inchstones, the percent complete for the GHI milestone = 31/39 = 79%

7.10.4. What Others Have Learned Merging Agile and EVM

Often we can learn from what others have learned. The following lists lessons organizations have mentioned when merging Agile and EVM. Our experiences have also validated the lessons of others.

Lessons Learned

  1. {color:#000;}Some programs are bid waterfall, yet executing Agile, meaning bids have not all caught up to Agile yet.
  1. {color:#000;}Cultural changes are often harder than technical changes.
  1. {color:#000;}The schedule should only go down to the level of Features (not to the story level).
  1. {color:#000;}Hybrid Scrum works, so tailor to your needs.
  1. {color:#000;}When delivery teams are one of many (scaled agile), it can be challenging to accurately reflect progress to “done” for a large portfolio.[30]
  1. {color:#000;}Agile EVM accurately informs both the Productivity and Predictability Performance Measure Categories.[30]
  1. {color:#000;}Baseline Cost Per Story Point and track every iteration, and performance thresholds should be based on acceptable variations.[30]
  1. {color:#000;}Organizations vary on when to credit user stories in EVM. The 0/100 EV technique works well. Other possible crediting of stories includes:

– 0% at start

– 80% at story completion

– 20% at customer acceptance (accounts for changes needed)

As with other improvements, we can apply lessons learned and continue to get better at applying EVM and Agile together. The following quote indicates that getting work done faster by using Agile might be just what we need going forward.

[_ "Our conventional programs seek a 99% solution in years. Missions require 75% solutions in months." [ 31]_]

— Robert Gates, United States Secretary of Defense, 2008

[]8. Adjusting to an Agile Mindset[]

[]8.1. Don’t Wait for Perfection

From a business perspective, getting some gains from a partial implementation of Lean-Agile is better than waiting until the conditions are ripe for a full implementation.

We have seen some articles in the software domain where they talk about needing Enterprise adoption of Lean-Agile. That is great if you can get there, but don’t wait until you can implement it in your entire organization.

Lean-Agile is good enough to provide gains even when partially implemented, for example in only a single project. Sometimes people at the level of the Training Department or the Learning & Development Department do not have the connections to pull off an enterprise-wide culture change. You don’t have to wait for enterprise-wide adoption to gain some benefits from Lean-Agile.

For buyers, you’ll need to find a contractor/supplier that you think can do it and then iterate.

Don’t let perfect be the enemy of good. General McClellan drove President Lincoln to frustration with this method of waiting for the perfect situation before doing anything during the American Civil War. It was not a successful strategy for him. The President replaced him. Apply what you can, get some early successes. Use that data to help justify and drive the expansion of the Lean-Agile approach.

Think experimentation using PDCA. Get leadership support for buying this way.

We have experienced gains implementing Lean-Agile inside a large organization (with tens of thousands of employees) without waiting for full Lean-Agile at the Enterprise level. You can achieve gains without having the entire parent organization do what you want to attempt.

[]8.2. Leading a Lean-Agile Implementation

[]8.2.1. Convincing Organizational Management

Much of how to do this depends on your leaders themselves and what they focus on. Convincing them can be easier when there is organizational pressure to speed up the development of learning experiences.

For buyers, even with smaller procurement budgets, you still have to assure stakeholders you will get them what they need. With Lean-Agile contract provisions, you can assure them their highest priority Learning Objectives will be satisfied during the project by using prioritization.

Use the time savings examples reported from software groups that have successfully applied Lean-Agile if you don’t have your own metrics to help persuade your organization to explore Lean-Agile. This could be groups inside your company, or groups external to your organization. We have seen significant savings using Lean-Agile.

Seek a quick initial win with a stacked development team of highly skilled people to help your leadership see that you can achieve success quickly. Share the successes. Add learned risk mitigation to your next project/program.

Don’t pick people for the first attempt that have a history of aiming critiques like Eeyore from the side lines. Pick those for the first effort that adapt well to change and can be enthusiastic about applying new ideas.

[]8.2.2. Asking Development Team Members to Change to Lean-Agile

Because we’re talking about humans, it is easiest to hire new development teams and set their expectations for using Lean-Agile. However, you can also convert existing employees into using Lean-Agile, who are used to other ways. We’ve done both successfully.

When you need to ask existing teams to make the change, it is a change management process. Pick the change management process you like if your organization does not have a favored one. Follow it. Message often. It takes time. Some people will adapt quickly. Others may take more time to adapt. Coach development teams during daily synchronization meetings. Coach buyer stakeholders during iteration demos.

[]8.3. Workplace Training Rather than Education

This book focuses on workplace training. We start from an engineering perspective, where the parameters of problem solving mostly came down to physical phenomena over opinion. A certain type of steel, for example, fails at a certain point in destructive testing. This is a data-driven conclusion rather than strong opinion. This informs our approach to this book. We have been involved in the occupational side of the training industry for many years now and have listened to and read many arguments between occupational training and education proponents. We focus on the more narrow workplace training over general education requirements because that is where we apply it.

Whether or not Lean-Agile may be able to help education is outside of the scope for this book. So, we will skip any arguments about what works in education. We are only currently addressing practitioners in workplace training for this book.

In companies with profit motive, the search is always on for more efficient or effective means to the end. The end in this case is an employee performing a skill that is typically obtained by the means of a training product that is deliverable to the customer and used for training employees on specific job performance.

Educators are welcome to use these ideas, but we did not make any attempt to optimize the ideas for their domain.

[]8.4. Lean-Agile Modifies ADDIE Rather than Replaces ADDIE

Sometimes people view ADDIE (analyze, design, develop, implement, evaluate) as both an instructional/LX design process and a project management approach.

Figure 39. ADDIE with Up Front Planning to Start (PADDIE)

We propose that ADDIE contains components that are still necessary for comprehensive instructional/LX design even in a Lean-Agile environment.

Viewing Lean-Agile as a project management approach or an operations management approach may help better communicate how we see the two sets of ideas merging. This is why we have made a mashup of both Lean-Agile and ADDIE. ADDIE has proven components for thorough instructional/LX design. Some people argue strongly when ADDIE is discussed as the entire approach for managing the project.

Figure 40. Where Lean-Agile Fits in ADDIE

The steps of the development process on our Kanban boards include ADDIE steps that apply on the particular project. For example, sometimes Analysis projects are performed as completely separate projects for many of our customers, so we won’t include analysis steps on our design and develop project Kanban boards. Conducting or facilitating courses is not wanted by some of our customers, so we leave off the implementation phase of ADDIE on our Kanban boards because that customer will have their own instructors conduct the sessions rather than our staff.

Operations management is a way of getting outputs from a system of people and processes. Project Management has a similar focus.

Targeting our audience is important in courseware and also in organizational messaging. Internal senior organizational leadership often have a background in engineering, operations, or a sub-specialty of HR rather than training. Communicating with them requires considering that they may not be as conversational on what ADDIE means as are training professionals using it in daily work. Messaging to leadership may be better received in the language of operations management and project management, rather than the language of specialists in learning experience development, depending on your circumstances. Even hardware and software engineers often need to translate their technical lexicon into management-speak to effectively communicate with that audience.

So rather than attempting to trigger dogmatic arguments about Lean-Agile versus ADDIE, we propose that the family get the new Lean-Agile sports utility vehicle in addition to the reliable ADDIE car. Having both in the driveway helps the family do more. It does not necessarily have to be a one-or-the-other discussion.

Additionally Lean-Agile is not the single unifying theory for instructional/LX design excellence, but rather a set of principles that inform your development of specifically tailored practices that your organization can use to be more productive, which improves profit margins and potentially velocity—​in this context meaning how many times you can earn your margin. When you can translate technician speak to financial impact you can more easily get your C-level (Chief Learning Officer [CLO], Chief Executive Officer [CEO]) leadership’s attention.

So rather than stir up passionate debates about whether ADDIE or Lean-Agile should be used, let us presume both apply and move on to how to execute training development projects with both.

[]8.5. Lean-Agile is Not a Silver Bullet

Some people may portray Lean-Agile as the fix to all problems. Carefully manage expectations.

Lean-Agile will not fix all problems. Many typical project management challenges remain with this approach because humans are involved and complex adaptive systems are at play.

Be careful that you don’t make it out to be the end all and be all of projects, or you may find yourself defending why they should let you use it again if you have problems. Most courseware projects have problems.

[]8.6. The Cottage Industry vs Mass Production Model

We read an article by JoAnn Hackos years ago that compared and contrasted technical writing as a cottage industry versus as a mass production model. We think her idea extends to learning and development organizations too.

The following table helps compare and contrast her two models.

Table 6. Cottage Industry vs Mass Production, by JoAnn Hackos [32]

Cottage Industry Model|=~{color:#000;background:rgba(0, 0, 0, 0);}. Mass Production Model
Dominated by a few craftspeople who know the craft and have great skill Large, multi-disciplinary teams, less experienced individuals have roles
Work independently, isolated Work as teams, collaboration
Slow training of few apprentices Train many people quickly
Innovation is largely a matter of individual preference Innovation is a matter of organizational decision making
Individual definitions of process, workers doing what they believe is valuable Organizational standard processes and practices, enforcement of standards
Craftspeople can be independent and work alone with a few assistants Everyone must know what everyone else is doing and work together

If your shop is small with low throughput of learning experience development projects, you may be in the cottage industry model and working it successfully. If your training product procurement or development is high throughput then you may need to use the mass production model to quickly scale.

We have seen some organizations that had a medium-sized team, and their leadership asked them to double their output and staff. The cottage industry model failed at that point. We have seen other organizations that had to triple the staff quickly. Again, the cottage industry model was not sufficient to support that speed of growth.

From the organization’s perspective, there is also increased risk in the cottage industry model if one or more of your key, highly-skilled craftspeople leave the organization.

The mass production model does not have to limit creativity. Think of the standards as similar to how an English Style Guide helps many training developers create a more consistent product or how a Code Style Guide helps many software developers create a consistent and readable code base.

[]8.7. Adopting Lean-Agile – Quickly or Slowly

[]8.7.1. Quickly–Using a Consultant

We have seen some software organizations move to Agile quickly by bringing in an outside consultant to coach their team into doing it correctly. This can be a fast approach and works if your organization has the money to pay the consultants. It seems, however, that you’ll need someone in-house that knows the Agile and Lean principles to help sustain the transition after the consultant leaves. Simply following a recipe will not work as well as understanding why it works.

[]8.7.2. Slowly–Do It Yourself (DIY)

The other way is slower, has less risk, is potentially less expensive, and may help you gain more expertise before you face the challenges of scaling while trying to apply this approach with the larger organization. First, read books and blogs from others currently trying it. Next try it on a micro scale, testing pieces of the approach, and encounter some problems where the practice for the exemplar organization does not work for you. Then, study the principles behind both Agile and Lean to better understand why what you have done didn’t always work as expected.

Then try again on a small scale. Use the PDCA cycle to improve. Learn more and study more about solutions to problem you experience. Adjust and try again. For example, initially we were unsure about how to interface Agile with Earned Value Management, and it took a while to test those ideas.

Then go for a larger rollout of Lean-Agile. First, with a single team of strong staff. This may be bumpy initially, but then it gets better. Continue to improve, striving to perfect some of the adaptations you made to the software version of Agile for use in learning experience development.

Next, expand Lean-Agile to multiple teams doing multiple projects. Continue to refine as you learn what sounds better on paper than works in a live development team.

Even internally, we have seen the need to have an in-house Lean-Agile Coach stay involved to help the teams apply the approach consistently.

Whichever way you choose, work hard at it and use the PDCA cycle to test each hypothesis against the experimental evidence. Then adjust and try again.

[]8.8. A Lean-Agile Journey

Our journey began by listening to others trying to communicate about how Agile could help with courseware.

One influence was Michael Allen. At an industry conference we heard Michael Allen talking up his book, Leaving ADDIE for SAM which describes his Successive Approximation Method (SAM). We decided to look into SAM a bit more. We found his iterative SAM approach interesting and helpful, but his book did not really provide enough information for us to build our own processes from scratch internally.

Around the same time, at a different conference about Darwin Information Typing Architecture (DITA) eXtensible Markup Language (XML), we heard reports of a handful of organizations using Agile. They supported software development teams that were using Agile. So we started researching Agile and noticed that Lean applies to it as well.

We had already been applying Lean as a company, and we started seeing the connections between Agile and Lean. In addition, the various Agile method proponents argued for their Agile way to be followed. Yet we find training learning experience development to be different than software development in significant ways. We could see that some of the practices being pushed could help us quickly, but others would take more time.

We started trying to apply Agile Kanban first in a smaller office because it required so little organizational change to get started. It could be done locally without authorizing additional funding. We also had concerns about how Agile would be perceived since it began life in the software development domain, and not from training.

Kanban was easy to see how to set up a white board, but it was initially hard to visualize based on what is written about it because it is about tracking a dynamic process. We tried to apply it with a team that was already started on a project, and we learned lessons.

We work in an earned value and traditional project management environment with detailed process practices.

We experimented with what was really needed on Kanban cards in our circumstances and what we could leave out.

Your experiences will vary depending where your organization is on the spectrum of formal processes. Some organizations range from near anarchy of organizational processes to near tyranny of such processes that discourage innovation.

Then we noticed our software engineers were already using Scrum and so we spent time trying to apply it to learning experience development. We found that parts of Scrum worked best for learning experience development and parts of it did not help as much given our set of circumstances at the time. Then we found others who had found both and combined the two into Scrumban. In the details of Scrum and Kanban, we found the specifics we needed to experiment and build out our working processes after the seeing Michael Allen present some of his iterative ideas from his Successive Approximation Method.

Some projects required a single prototype, so iterations of prototypes seemed out for that circumstance. Our courseware design documentation (CDD) was still firmly an expectation with Government buyers and therefore a requirement, and yet the “iterations” for design tended to only be resolving customer comments from their review of the document without getting to courseware. We cover more on Lean-Agile Design later in this book.

Later on, we worked with some customers with a more iterative design approach too, but we only applied Lean-Agile to development initially. We did not have the option of only performing rapid prototyping, so parts of SAM would not work for us.

At an eLearning Guild conference, formerly called mLearnCon, we heard from a handful of other organizations that were applying Lean-Agile to their training development. We listened as they shared what they did. Most of these groups supported software product development groups where the organizational culture had already accepted Agile for software development and they were applying it in that context. Their circumstances made for an easier persuasion effort to get organizational leadership buy-in or customer buy-in.

One lesson was that you don’t learn it all in one giant effort, but learn a little here and a little there. Confidence in executing Lean-Agile has to be learned from doing it. The most successful ways of communicating it to influential team members, internal leadership, and buyer stakeholders has to be mastered in parallel. Technical expertise alone is not enough to facilitate a large organizational change management effort. Communication plays a big part. Listening to detractors and finding answers to their objections was important. Listening to team members struggling to learn it and apply it has led to insights about how to modify our training for team members and to some modifications of our current methods. We continue to learn and improve.

Another lesson is to measure the Lean-Agile approach from start to finish and compare metrics.

[]8.9. Driving Assumptions, Context and Constraints

The principles of Agile and Lean can be applied in an infinite variety of ways depending on your situation today, and how your situation changes as you learn. As you apply the principles of Lean-Agile in your own context, it may be helpful to know the context assumptions and constraints under which the ideas in this book were formed.


  1. {color:#000;}Make training for other organizations, more so than for our own employees.
  1. {color:#000;}Training is “owned” by engineering rather than HR.
  1. {color:#000;}Government customers often require the use of earned value management on all training projects. Therefore, the tracking described may be more involved than you need.


  1. {color:#000;}We often put instructional/LX designers into team lead positions. They know the learning experience development process well and can help focus the team in the right ways.
  1. {color:#000;}We have various delivery means, such as instructor-led training, eLearning, 3D immersive experiences using game engines, and simulations.
  1. {color:#000;}We include formal ADDIE evaluations when the customer requires and pays for that.
  1. {color:#000;}Some projects applied Lean-Agile as a new process and have included minimal interfaces back to the rest of the customer organization that still uses sequential waterfall process with decision gates and big-requirements-up-front (BRUF) planning.
  1. {color:#000;}Rather than an all or nothing approach, we use as much Lean-Agile as currently possible and gradually apply more of it as successes help fend off any organizational ‘white blood cells’ (people conditioned to the waterfall processes and that call out and sometimes resist any change from those processes).


  1. {color:#000;}Some customers prefer not to use Lean-Agile. Reasons vary. We don’t force it.
  1. {color:#000;}Some organizations we work with still use more waterfall methods than Lean-Agile methods, and their organizational leadership is more familiar with waterfall approaches to project management. This causes groups applying Lean-Agile to explain and persuade more as they go.
  1. {color:#000;}Changing organizational processes can take months of effort with large organizations.

Your organizational context, assumptions, and constraints will surely be different, so adapt the principles of Lean-Agile to your needs.

[]8.10. Lean-Agile is Still New to Many Training Product Buyers

Lean-Agile is still new to many buyers/customers, both internal and external, wanting courseware. You may be able to influence them into applying Lean-Agile. You need to aim for a relationship with customers that is strong enough that they will give you a hearing if you propose why this approach can help them. It helps you too, and is win-win, but many customers prefer to hear primarily how Lean-Agile helps them.

Some buyers are open to it if you can show them how it will help them. Some are opposed to changing to some new process when they use the waterfall method with all of their other stakeholders and suppliers. Some may not feel they have the political capital to lose if they view your proposal as an experiment and worry you may not succeed. In situations where you do more than submit a written proposal, do more to persuade them. Ideally, meet face-to-face or get on the phone. Asynchronous messaging with email and text messaging is insufficient for this persuasive task. We have had many customer discussions in person and on the telephone about how this can help them and how it will impact their subject matter experts (SMEs).

The buyer/customer individuals who are open to Lean-Agile also need to ensure they come through for their own boss and stakeholders, so as long as they see consistent progress, this can work. The iteration demonstrations are trust-reinforcing as the customer sees all the plans turned into working courseware increments earlier in the process than they might have seen otherwise.

The software industry is awash with Agile concepts after 10+ years of use, and it seems that most everyone in the software arena is familiar with Agile to some degree. However, the training industry did not come up with this idea, and movement to Agile is occurring more slowly. So even when you get your buyer/customer to agree to Lean-Agile, you may still need to help them during the periodic reviews with their own management and stakeholders to help them understand why you’re using this approach and how it helps them.

Because of the many proponents pushing various specific Agile methods rather than focusing more on the principles, your buyer/customer may also have some misconceptions about Lean-Agile that you will need to listen for and address appropriately.

[]8.11. Complexity Impacts Training Projects Today

Nonlinearity and complexity would be nice to ignore, but they impact learning experience development projects. This section gets to the meat of the challenges today. Though you may be tempted to skip this section, consider that complexity is a large part of why Lean-Agile works well in our modern courseware projects.

By analogy, if you needed to create eLearning or mobile learning but had no authoring tools besides Microsoft Word and PowerPoint, you would face many hurdles. Taking some time to investigate appropriate authoring tool options and selecting a good one can reduce your frustrations and help you reliably deliver training products on time. Similarly, taking some time to investigate how complexity impacts training projects today and exploring how you can to respond differently can mitigate risk and reduce frustration.

Before we look at the relevance of complex systems to learning and development, let’s clarify a few terms.

When discussing complexity it is helpful to also discuss systems and their boundaries. A system is a set of interacting or interdependent component parts forming a complex/intricate whole.[33] Every system is delineated by its spatial and temporal boundaries, surrounded and influenced by its environment, described by its structure and purpose and expressed in its functioning.[33] The term system may also refer to a set of rules that governs structure and/or behavior.[33] We may choose the system perspective to consider the courseware, the buyer’s team, the producer’s team or a particular technology component.

Our discussion of systems necessarily includes complex systems. The study of complex systems represents a new approach to science that investigates how relationships between parts give rise to the collective behaviors of a system and how the system interacts and forms relationships with its environment.[34] It is useful to represent such a [complex] system as a network where the nodes represent the components and the links their interactions.[35]

Modern scientific study of complex systems is relatively young in comparison to conventional fields of science with simple system assumptions, such as physics and chemistry.[35]

Complexity science says that we’re surrounded by systems and that these systems are complex and constantly adapting to their environment—called complex adaptive systems.[36]

Now that we have the terms clarified, let’s look at the relevance of complex systems to learning and development.

LX development teams and groups of buyer stakeholders are complex adaptive systems with elements, in this case people, aggregated into systems, in this case teams or groups, that demonstrate complex behavior and adapt to a changing environment around them.

For much of our history, we have viewed the world and universe around us as a linear environment where simple cause and effect rules apply. Mankind has believed for most of our history that everything could be predicted and controlled.

We have often used our preferred linear mindset, looking for causality—that connects one process (the cause) with another (the effect)[37]—in our collective efforts to better understand and predict the behaviors of the systems around us to aid our planning, problem solving, and decision making. We often rely on decomposing or breaking systems down into smaller components to understand the pieces apart from the whole. We call this approach systems analysis. This is closely related to requirements analysis.[38]

Slowly, we have begun to better understand complex systems and the phenomena associated with complex systems. As theories about complex systems have begun to emerge, people from many disciplines have discovered similar findings.

The search for simple solutions to complex problems is a consequence of the inability to effectively deal with complexity.

— The late Russell Ackoff

The challenge in gaining a more full understanding of system behavior by the traditional decomposition approach with complex adaptive systems is that the components interact differently.

Complex systems may have the following features:[39]

  1. {color:#000;}Possess a high amount of structure or information, often across multiple temporal and spatial scales.[40]

  1. {color:#000;}Complex systems may have nonlinear interactions.[39] Small perturbations may cause a large effect, a proportional effect, or even no effect at all.[39] Effects may be indirect rather than linear cause and effect.
  1. {color:#000;}Easy to see in hindsight, but very difficult to see in foresight.[41]
  1. {color:#000;}Complex systems have a large numbers of components, or autonomous processing nodes, [42] often called agents, that interact and adapt or learn.[43]
  1. {color:#000;}Dynamic systems of interacting agents.[44]
  1. {color:#000;}The agents change the state of their environment by their actions.[42]
  1. {color:#000;}Complex systems may have a high degree of adaptive capacity, giving them resilience in the face of perturbation.[36] The agents as well as the system are adaptive.[43]
  1. {color:#000;}Complex systems may have cascading failures.[39]
  1. {color:#000;}Due to the strong coupling between components in complex systems, a failure in one or more components can lead to cascading failures.[39]
  1. {color:#000;}Complex systems may be open systems.[39] It may be difficult or impossible to define system boundaries.[43]
  1. {color:#000;}Complex systems have a history [43] or a memory.[39] They evolve and their past is co-responsible for their present behavior.[43]
  1. {color:#000;}Complex systems may be nested.[39] The agents of a complex system may themselves be complex systems.[39]
  1. {color:#000;}Complex systems may have networks of agents with many local interactions.[39] Interactions are primarily with immediate neighbors and the nature of the influence is modulated.[43]
  1. {color:#000;}Complex systems may produce emergent behaviors.[39]
  1. {color:#000;}In complex systems, relationships contain feedback loops.[39] Both negative (damping) and positive (amplifying) feedback are always found in complex systems.[39] This is known as recurrency.[43]

We recognize more nonlinearities in today’s world, including in learning and development. Unexpected events, volatility in requirements, technical complexity, social complexity, organizational culture and interdependence between parts of a courseware project have all added uncertainty to projects, and they tend to cost more and take longer. Bottlenecks that aren’t noticed for too long impact costs and schedule.

Overemphasis on efficiency can make our teams and processes more fragile to random stresses that occur unpredictably. When applying Lean, be careful not to reduce organizational structure to the point where it cannot absorb unexpected shocks and survive. Buffers, reserves, and redundancy help us avoid fragile structures. Our projects tend to get as weak as the weakest link in the chain. The Theory of Constraints buffers the primary constraint.

Parent organizations and buyer stakeholders still desire predictions, but often predictability is elusive in complex projects.

Complexity theory predicts that we cannot rely on predictions.[45]

— Jurgen Appelo

Lean-Agile provides an approach, rather like an exploratory model, that breaks the project size down into smaller pieces, experimenting, and gaining feedback and adjusting earlier in the project lifecycle, which reduces uncertainty each iteration, increasing our chances of a successful project. Feedback changes the system. Visible Kanban boards allow you to see bottlenecks as soon as they appear, reducing delays, which in systems dynamics tend to cause overshoots in our responses if we’re acting on outdated information. Daily synchronization meetings provide information exchange and collaboration instead of standard status reporting meetings. A Lean-Agile team’s size caps out at 7-9 people ideally, reducing the impact of size effects. Smaller experiments by more teams produce smaller harm when they fail than do huge experiments that can become too big to fail.

Feedback is an important part of Lean-Agile used in teams that are complex adaptive systems. Each team member observes the daily flow of information and context on the Kanban board and responds appropriately. Their individual responsive actions consequently feed back and influence the next set of contexts for the team. Damping or balancing feedback tends to help settle on the desired target change, while amplifying or runaway feedback tends to reinforce the trend, sometimes leading to nonlinear snowballing and exponential changes. Delays in feedback can cause actions that cycle between overshoots and undershoots to the target change. Daily standups and Kanban boards making information visible to the entire team help mitigate delays in feedback.

Attempts to decompose a complex system into its component agents and observe the one agent’s behavior in an effort to predict the system’s behavior are obstructed by the tendency for emergent system behaviors. Emergence is a process whereby larger entities, patterns, and regularities arise through interactions among smaller or simpler entities that themselves do not exhibit such properties.[46] The emergent property itself may be either very predictable or unpredictable and unprecedented, and represent a new level of the system’s evolution.[46]

For LX development teams, small issues may accumulate and may cause large effects or even blowup the project/program. Or the accumulation may have no effect at all. Each person in a development team may not have all information about the project, although Lean-Agile mitigates this somewhat. People communication and collaboration leads to unexpected results.

Even as learning and development professionals, we strongly encourage gaining a deeper understanding of these complex adaptive systems because it will illuminate some of why you have experienced what you have experienced. It also can help you to see why Lean-Agile principles can help address our challenges going forward.

We also recommend Nicolas Taleb’s books about the effects of uncertainty, volatility and randomness. His titles are Black Swan, Antifragile and Fooled by Randomness. His books are dense with original thinking that can also apply to learning experience development. The experimental and adjustable nature of Agile fits the optionality that Taleb recommends.

We humans work hard to develop mental models or maps to navigate our world. Be careful not to confuse the map for the actual terrain or the model for the reality. Our maps and models are incomplete simplifications to help us find our way. Thinking about complex systems depends on context.

Multiple weak models can make just as much sense as one strong model (and its certainly better than no models). In the end, all models fail.[47]

— Jurgen Appelo

So does our adaptation of Agile principles address complexity? Lean-Agile:

  1. {color:#000;}Addresses complexity with complexity

  1. {color:#000;}Uses multiple models, rather than over dependence on only one model like Scrum or ADDIE alone
  1. {color:#000;}Depends on context
  1. {color:#000;}Allows for subjectivity by system participants and the impact of their actions on the system

The principles of warfare that we mentioned at the beginning of this book make no mention of “best practices”. David Snowden’s Cynefin Framework identifies best practices as only appropriate for his “obvious” domain which would include simple courseware projects. Emergent practice is more appropriate for warfare (complex and chaotic domains) and for complex courseware projects. Lean-Agile supports emergent practice. Adapt a principle to your situation rather than copy someone else’s method(s) without changing them.

Your organization may still crave predictability. You will have to work within that challenge.

Be careful applying any one expert’s opinion or mental model as a universal solution in our complex world. Apply correct principles into current methods. Adjust your methods as circumstances change. In the end, we point back to small scale experiments, which cause little harm when they fail, using PDCA to continually improve. The plan part of Agile is both backward-looking, last iteration, and forward-looking, imagining an ideal future state. Find what works for you in your context and apply more of what works and less of what doesn’t, just like the children’s game of red-light, green-light.

This also means that teams not accustomed to identifying and quickly and often applying lessons learned tend to need coaching to realize that the adjustments to the feedback are where most of the gains of Agile come from.

Our purpose for adding a section about complex systems in a book about learning experience development is to highlight the need to understand indirect effects when planning and executing courseware projects/programs. Start your problem solving using the Cynefin framework to help you realize where your project fits.

Applying the Cynefin Framework for Training Projects/Programs

  1. {color:#000;}If cause and effect is obvious to all then you’re in the obvious domain where best practices can work well as you sense, categorize, and respond.[48]
  1. {color:#000;}If the relationship between cause and effect requires analysis or investigation and/or expert knowledge then you’re in the complicated domain where good practice can work as you sense, analyze and respond.[48]
  1. {color:#000;}If the relationship between cause and effect can only be perceived in retrospect, but not in advance, then you’re in the complex domain where emergent practice will see you through as you probe, sense and respond.[48]
  1. {color:#000;}If there is no relationship between cause and effect at the systems level, then you’re in the chaotic domain and only novel practice will see you through as you act, sense and respond.[48]

Solving problems within complex systems can be challenging and difficult. Assuming linear cause and effect is only appropriate in the obvious and complicated domains. As you influence your organizational culture towards adapting to complexity, Lean-Agile will help.

[]9. Principles of Agile – The Key to Success[]

[]9.1. Principles Versus Methods

“As to methods there may be a million and then some, but principles are few. The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble.” [49]

— Ralph Waldo Emerson

Having seen business process reengineering in multiple organizations, and watching the fervor that some have for a particular set of methods like Scrum, it seems some easily forget that the principles of Lean-Agile mean we can adjust the methods as necessary. Rather than a set of practices or methods cast in concrete, fluency with the principles can help teams adapt as needed to remove their impediments and improve their team throughput and quality with each iteration.

Principles focus on the why, while practices focus on the how. This book sets out a few practices that worked for us in specific circumstances that may need tailoring and adaptation for your circumstances.

This book is more about the application of Agile and Lean principles to learning experience development. Although much is known about these principles, not all organizations have been successful in applying them because the thinking required to apply these principles requires a change in organizational culture, behaviors, and practices that may be beyond what some organizations can do. Much depends you how you lead any effort into this realm and how you interface with the parent organization if they are not joining you on the Lean-Agile journey just yet. If you…​

  1. {color:#000;}lead the transition well

  1. {color:#000;}adapt the principles to your context so your practices work operationally in your conditions
  1. {color:#000;}teach everyone involved why the principles work

…​then, you too can achieve surprisingly significant improvements.

[]9.2. Agile Principles

[]9.2.1. Agile Manifesto

It may help to review the Agile Manifesto [50] (modified slightly for training) that articulates principles. These principles were articulated by the Agile Alliance.

We are uncovering better ways of developing software [courseware] by doing it and helping others do it.[50] Through this work we have come to value: [50]

  1. {color:#000;}Individuals and interactions over processes and tools [50]

  1. {color:#000;}Working software [courseware] over comprehensive documentation [50] [of intent]
  1. {color:#000;}Customer collaboration over contract negotiation [50]
  1. {color:#000;}Responding to change over following a plan [50]

That is, while there is value in the items on the right, we value the items on the left more.[50]

Caution When you plan new efforts where experimenting and failure will be necessary, the plan is the best guess with what we know at the time we write it. To pretend it is more than the best guess at the time increases risk.*]
Note*][_ In some organizations, the development teams have no control over contracting with the customer, so they may not be able to apply “Customer collaboration over contract negotiation” at all or not perfectly. This is where the application of the principle will allow you to come up with a practice that works for your context._]

[]9.2.2. Agile Principles

The site http://agilemanifesto.org/principles.html states the following principles for Agile. At this point in the book we will simply restate them as originally articulated by the Agile Alliance at agilemanifesto.org with only slight replacement of software with courseware.[50]

Principles behind the Agile Manifesto [50]

We follow these principles: [50]

  1. {color:#000;}Our highest priority is to satisfy the customer through early and continuous delivery of valuable software [courseware].[50]

  1. {color:#000;}Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.[50]
  1. {color:#000;}Deliver working software [courseware] frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.[50]
  1. {color:#000;}Business people [customer stakeholders and SMEs] and developers must work together daily throughout the project.[50]
  1. {color:#000;}Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.[50]
  1. {color:#000;}The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.[50] [When geographically dispersed, we adjust and use conference lines. Face-to-face is the ideal.]
  1. {color:#000;}Working software [courseware] is the primary measure of progress.[50]
  1. {color:#000;}Agile processes promote sustainable development. The sponsors, developers and users [, stakeholders, SMEs and reviewers] should be able to maintain a constant pace indefinitely.[50]
  1. {color:#000;}Continuous attention to technical excellence and good design enhances agility.[50]
  1. {color:#000;}Simplicity—​the art of maximizing the amount of work not done—​is essential.[50]
  1. {color:#000;}The best architectures, requirements, [content] and designs emerge from self-organizing teams.[50]
<{background:rgba(0, 0, 0, 0);}.
p((((((((((={color:#000;}. * Note*
<{background:rgba(0, 0, 0, 0);}.
p((((((<{color:#000;}. [_ We adjust this principle slightly to ‘directed self-organizing teams’ by giving people a clear purpose and defined goals and constraints as boundaries. Ex-military folks will see this as similar to the Commander’s Intent portion of an operations order. For those not familiar with this, it allows lower unit leaders the freedom from being told exactly how to get it done, and to change the way they respond to the situation so long as they help the parent unit achieve its objectives. This rephrasing of self-organizing tends to be more palatable in organizations._]
  1. {color:#000;}At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.[50]

[]9.3. Lean Principles

Lean is not a tactic or a cost-reduction program, but a way of thinking and acting. Lean thinking puts forward the idea of all of the processes forming a value stream, like a water stream, creating value that keeps flowing downstream to the customer as quickly as is sustainably possible with as few eddies, dams, or other wasteful slow downs or stops due to obstacles. The short list is:

  1. {color:#000;}Identify the current customer Value.

  1. {color:#000;}Visualize the whole product development process (value stream).
  1. {color:#000;}Reduce Waste along entire value streams.
  1. {color:#000;}Create Flow ( Batch Size Reduction).
  1. {color:#000;}Establish Pull (Just-In-Time Production).
  1. {color:#000;}Respect People.
  1. {color:#000;}Amplify Learning (we use Scrum Retrospectives).
  1. {color:#000;}Seek Perfection ( Kaizen, or get a little better each iteration).
  1. {color:#000;}Lead Lean.

The following longer list includes some clarification about what is meant by each principle.

[]9.3.1. Clarification of each Principle

Although readers experienced with Lean may perhaps find this next section repetitive, we include it for those in the training profession that are new to Lean because really grasping the principles is crucial to the effective use of Lean-Agile and tailoring your methods to fit your requirements. This section clarifies what is meant by each Lean principle.


Identify current customer Value

  1. {color:#000;}The customer will pay for it.

  1. {color:#000;}Perform work right the first time.
  1. {color:#000;}Specify value from the perspective of the end-customer.
  1. {color:#000;}Create more value for customers with fewer resources.
  1. {color:#000;}The customer changes over time, what satisfies them now may not later.

Value Stream

Visualize the whole product development process (value stream).

  1. {color:#000;}Figure out how the work gets done.

  1. {color:#000;}Use visual boards to display necessary status and progress information at a glance.
  1. {color:#000;}Optimize communication with frequent (daily) short (15 min) scrum/standup meetings in combination with the visual Kanban board.
  1. {color:#000;}Use visual signals and cues.
  1. {color:#000;}Anyone can see work in process (WIP) easily if they have access to the Kanban board.
  1. {color:#000;}Align the team through simple visual communication.
  1. {color:#000;}Apply queuing theory, see later section on Using Queuing Theory to Improve Kanban Traffic Jams.[51]
  1. {color:#000;}Visual project management has been described as the ability to understand the status of a project in 5 minutes or less by simple observation without speaking to anyone.


Reduce Waste along entire value streams.

  1. {color:#000;}Recognize the types of waste Lean identifies.

  1. {color:#000;}Whenever possible, eliminate those steps that do not create value.
  1. {color:#000;}Develop processes that require less capital, physical space, human performance and time.
  1. {color:#000;}Create the ability to respond to changing customer value quickly at low cost.
  1. {color:#000;}Make information management much easier and more correct.
  1. {color:#000;}Establish value as seen by the customer to distinguish value added from waste.


Create Flow ( Batch Size Reduction).

  1. {color:#000;}Create flow by reducing and eliminating waste.

  1. {color:#000;}Deliver as quickly as possible.
  1. {color:#000;}Interruptions in the process steps degrades the flow of buyer/customer value in the value stream—To achieve smooth flow, reduce interruptions.
  1. {color:#000;}Too much work in process (WIP) creates delay waste due to context switching.
  1. {color:#000;}Reduce batch sizes to reduce the amount of WIP inventory.
  1. {color:#000;}Cycle time is almost directly proportional to the amount of WIP. Multitasking is an illusion.
  1. {color:#000;}Smaller batch sizes shorten the overall production cycle time.
  1. {color:#000;}Inventory turns (velocity) increase as we shorten production cycles. More turns allow operating profitably at lower margins.
  1. {color:#000;}The ideal flow is “one piece flow” (finish one work item before starting another one).
  1. {color:#000;}Create a level or smooth development process flow (decompose work effort into similarly sized work items).
  1. {color:#000;}Assess the value stream to make sure each step adds value, can make the desired result, that resources are available and adequate.
  1. {color:#000;}Organize so all the expertise needed to produce the product/service is on a cross-functional team rather than in traditional (functional) departments.
  1. {color:#000;}Integrate suppliers in the product development process.
  1. {color:#000;}Synchronize any simultaneously executing processes.
  1. {color:#000;}The fixed iteration time box/tempo of iterations keeps a sense of urgency and effectively sets the team cadence.


Establish Pull (Just-In-Time Production).

  1. {color:#000;}Only pull the amount of work item cards you can do (pull less than or equal to the WIP Limit). Often this is 1–2 per person.

  1. {color:#000;}Be accountable for work items you pull.
  1. {color:#000;}Use pull-based OJT training for development team members rather than push-based training all at once.
  1. {color:#000;}Build the required work item when it is required.
  1. {color:#000;}Downstream customers pull value from the preceding upstream activity, setting the pace.
  1. {color:#000;}No one upstream in the process should produce a work item until a downstream customer asks for it.
  1. {color:#000;}Less time is wasted building unsalable product.
  1. {color:#000;}Reduce or eliminate bottlenecks.
  1. {color:#000;}Pull is sometimes described as just-in-time.


Respect People.

  1. {color:#000;}Focus on process fixes first before seeking to blame people on the team.

  1. {color:#000;}Make the pace of work sustainable indefinitely–too much overtime adds risk.
  1. {color:#000;}Use lessons learned during retrospective meetings–a culture of blame constrains innovation, while collaborating how to do better supports improvement.
  1. {color:#000;}Let the people who do the work figure out the improvements each iteration.


Amplify Learning.

  1. {color:#000;}Over time, teach Lean thinking and skills to everyone involved.

  1. {color:#000;}Use short iteration cycles to speed the learning process.
  1. {color:#000;}Learn what to improve by increasing feedback using short feedback sessions with customers.
  1. {color:#000;}Use the Lean language so communication is efficient—All involved learn it.
  1. {color:#000;}Seek progress allowing for some mistakes as you learn.
  1. {color:#000;}At the start of the product design process, thoroughly explore alternative solutions while there is maximum time available for design.
  1. {color:#000;}Prevent accumulation of defects by running quality assurance content checks and functionality testing as soon as the content is written or the functionality is working.
  1. {color:#000;}Share effective improvements between teams on large projects.


Seek Perfection ( Kaizen, or get a little better each iteration).

  1. {color:#000;}Speed up PDCA (Plan, Do, Check, Adjust) cycles.

  1. {color:#000;}Improve the way projects are executed.
  1. {color:#000;}Focus on root causes not symptoms.
  1. {color:#000;}Build a culture to support excellence and relentless improvement.
  1. {color:#000;}Use standardization to create broader flexibility.
  1. {color:#000;}Build in continuous improvement and team learning.
  1. {color:#000;}Build in quality. Design processes, products and services to eliminate errors rather than inspecting work at the end of the process to make sure we find all errors.
  1. {color:#000;}Tailor technologies to fit your people and process where possible.
  1. {color:#000;}Get everyone actively engaged in operating the value stream correctly per the latest improvements.
  1. {color:#000;}The team looks for any lessons from failures.
  1. {color:#000;}Continually challenge the status quo.


Lead Lean.

  1. {color:#000;}Change from command and control style of leadership to leadership based on questioning, coaching and teaching and rooted in the scientific method of PDCA experiments.[52]

  1. {color:#000;}Read the respect people section again.
  1. {color:#000;}Engrain respecting people into leaders at all levels.
  1. {color:#000;}Track numbers and manage by evidence.
  1. {color:#000;}Lean leaders lead from gemba, where the action happens.
  1. {color:#000;}Apply the 3 “gen” or actuals:

–genchi–(like gemba) go to the actual place.

–genbutsu–observe the actual product, process or service.

–genjitsu–gather actual facts.

  1. {color:#000;}Ask open-ended, probing questions.

  1. {color:#000;}Plan in more detail only when you get closer in time to doing the work (like rolling wave).
  1. {color:#000;}Unanticipated events occur during project life cycles.
  1. {color:#000;}Decide as late as possible, particularly for decisions that are irreversible, or at least will be impractical to reverse.
  1. {color:#000;}Minimize project administration.

Hopefully you can see how Lean and Agile are closely related with each other. It pays to understand Agile at the principles level before determining what method changes to incorporate.

[]9.4. Theory of Constraints Principles

Optimizing individual processes does not help throughput as much as optimizing for the entire process throughput. Rather than sub-optimizing at too low a level, focus on the bigger picture.

The biggest difference between the Theory of Constraints and Lean is the idea of subordinating local optimization to global optimization for the entire system’s throughput. Goldratt himself stated that the Theory of Constraints tells you where to look and what to change, while Lean tells you how to change.

  1. {color:#000;}Global optimization with local action (like Pull in Lean).

  1. {color:#000;}Find the constraint or bottleneck.
  1. {color:#000;}Decide how to handle the constraint using Lean.
  1. {color:#000;}Subordinate everything else.

The Theory of Constraints includes an idea it calls drum-buffer-rope. Briefly, this concept means the drum sets the pace or ‘beat’ of the entire work flow system. To meet the drum pace, we need to protect the constraint from lost capacity due to unexpected problems. We protect it using a buffer. However, buffers or queues can increase inventory. So to help protect against too much WIP inventory, we only want to start another work item after the main constrain finishes one. So a rope is figuratively tied between the constraint process step and the upstream starting process step to gate or choke the workflow system. This is similar to the Los Angeles freeway on-ramp example that stops cars from entering until the preceding cars have entered the traffic flow. The TOC drum-buffer-rope idea originated from manufacturing. We see the TOC drum-buffer-rope idea as the nearly the same as pull in Lean. We also find it tends to be easier to explain to knowledge worker staff involved in training, “Don’t pull another work item card until you complete one.” So going forward, we won’t mention drum-buffer-rope again. We’ll only talk about Lean pull instead.

We look for bottlenecks on our Kanban board as part of the daily scrum/standup meeting.

We recommend the book, Velocity: Combining Lean, Six Sigma and the Theory of Constraints to Achieve Breakthrough Performance – A Business Novel. It is written as a story to more effectively show the principles of the Theory of Constraints in action. It is aimed at manufacturing, so the reader will need to interpret from that domain to the training domain.

Applying the Theory of Constraints to avoid sub-optimizing helps an individual instructional/LX designer realize that simply improving their own personal workflow does not necessarily improve the overall throughput of the entire development team. Even when the instructional/LX designer is the primary constraint, there are still practices in Agile that can improve the overall team’s performance.

People are often not used to global optimizing when traditional metrics have looked at utilization of machines and people (sub-optimizing) rather than measuring improvement in the entire process throughput. We use TOC to set WIP limits.

Another challenge for training professionals is seeing all of our unfinished work products as “inventory.” Just like in manufacturing, increased inventory is waste.

[]9.5. Minimizing Feedback Delay

Systems thinking, or the study of system dynamics, offers a principle that helps explain why Lean-Agile has been so successful for our teams.

In systems thinking, there is the concept of feedback in a system. Controls in the system monitor the stock levels and take action to adjust the inflows or outflows. Delays in system feedback can cause oscillations in stock levels. A full discussion of systems thinking is outside the scope of this book, but for more information, consider Thinking in Systems, by Donella Meadows, 2008.

For Lean-Agile teams, the daily standup meeting reduces feedback delays to 24 hours. In a brief 15-minute meeting, each day, the Lean-Agile team can see the stock of work items on the Kanban board and adjust the flow of work items through the process steps as needed. In some organizations, there is only a monthly financial report on progress, increasing the delays. The delay can result in not having the project finish on time or it costing more than expected.

David Snowden’s discourses [53] on complexity science also help explain why the daily standup is needed because the development team is a complex adaptive system.

The Kanban board helps see the emerging patterns, and the daily standup lets the whole team see the situation, so they can offer ideas for a solution that could not have been envisioned before the situation happened. Our prior method of having the Lead maintain the project status in MS Project reduced visibility to only the Lead rather than the entire team.

Having weekly or monthly meetings adds significantly more delay 7X to 30X which leads to more issues. When people get worried about more meetings, assure them you will cut the meeting at 15 minutes and then follow through.

[]9.6. Make Work Visible

Knowledge work is not visible when you walk by a cubicle or office space. It is difficult to improve that which you can’t see. Training development is knowledge work.

Our Lean-Agile uses Kanban boards to make the work status and progress (its flow) visible while it is still in process. In addition to making the flow of the work visible, it also makes the process policies explicit and visible for everyone, allowing those involved to consider quality during the work rather than tacking quality on at the end.

  1. {color:#000;}Visibility then enables the collaboration of the team members to continually improve in what Scrum calls retrospectives.

  1. {color:#000;}Visual management tools help team members observe WIP accumulating and bottlenecks as they form.
  1. {color:#000;}Visualizing the effects of Lean wastes is the first step to reducing or eliminating them.
  1. {color:#000;}Visual management helps members of the team collaborate during the daily standup without a supervisor having to update a planning spreadsheet or Gantt chart first, reducing transaction waste.
  1. {color:#000;}The Kanban board clearly shows the next highest priority work item for team members to pick, reducing the need for a supervisor to direct each task after a team member completes their prior work item. This also reduces transaction waste.
  1. {color:#000;}Visual management reduces the need for supervisory intervention down to exceptional escalations rather than as standard work.

Lean visual management helps achieve the Agile principle of self-organization.

[]9.7. Geographically Dispersed Teams Versus Colocated Teams

We sometimes have physically colocated teams making training, but for other projects we’ve drawn on talent from across the United States.

While learning from the literature about software teams’ use of Agile, many pushed for colocating teams. We agree that whenever possible this is the highest bandwidth interaction method for teams to collaborate and build products together. However, for our current conditions, the choice is not as simple and is influenced by many other factors.

[]9.7.1. Applying Lean-Agile to Colocated Teams

Applying Lean-Agile to colocated teams is cheaper if you can find a wall space for your Kanban board. The budget is for office supplies. The following is a partial list of what you will need:

  1. {color:#000;}Locate a whiteboard, cork board, or large glass window.

  1. {color:#000;}Find enough room near the board for all the team members and any stakeholders to attend standing up.
  1. {color:#000;}Draw columns the board that represent your workflow.
  1. {color:#000;}Get lots of sticky notes.
  1. {color:#000;}Get wide tip markers, so the team can see the card text from 3-6 feet (1-2 meters) away.
  1. {color:#000;}Ask team members to identify avatar pictures to attach to work item cards.
  1. {color:#000;}Set a team rule that team members must update the position of their work item cards on the Kanban board at least daily before the standup meeting.

[]9.7.2. Applying Lean-Agile to Geographically Dispersed Teams

Applying Lean-Agile to geographically dispersed teams is functionally the same, but the tools are different. A partial list of the tools you will need follows:

  1. {color:#000;}License a virtual Kanban board ($5-$15 per month, per user), ideally one with batch import of work items from a spreadsheet, automated analytics and with activity history that has a timestamp of when each card moved to each column (work step).

  1. {color:#000;}Configure the board with columns representing your workflow.
  1. {color:#000;}Upload team member avatar images to the virtual Kanban board app.
  1. {color:#000;}Set a team rule that team members must update the position of their work item cards on the Kanban board at least daily before the standup meeting.
  1. {color:#000;}Find a way to conference call all the team members and any stakeholders attending.
  1. {color:#000;}Either have everyone log in at the same time or use a screen casting app to share one view to all.

All the normal daily standup actions apply to both types of team.

[]9.8. The Seen and the Unseen – Noticing Lean Waste

Most of the Lean body of knowledge is directly applicable to learning experience development. However, some parts of it do not transfer without a change in thinking because LX product development is different from manufacturing. Team members can help identify waste when they know the expectation. This section is intended to help see what may be unseen without the Lean mental filters engaged.

Most learning experience development involved worker information flows and processes are invisible to someone walking around the gemba, or work place. Lean emphasizes visual management. This can help us see waste that typically remains unaddressed in the unseen flows and processes.

Waste means anything—​time, costs, work—​that adds no value in the eyes of the customer.

Lean typically identifies the following types of waste:

Waiting, Delay, Queue Time waste.

This includes people waiting for anyone or anything. Examples include waiting for available resources like SMEs, internal process steps, data and responses from people. Waiting for SMEs is waste. Although not often thought of as waste by knowledge workers, WIP inventory waiting to be worked on is also waster. Teach teams to see All queue time as delay, meaning waste. Improvements seek the causes. Track queues on your Kanban board to visualize it so you can reduce it. A queue distinguishes work that is eligible to be pulled from work that is still in process. Team members waiting for information, instructions and tools all are waste too. Waiting for the hiring process of team members to finish is delay waste that may cause an imbalance between supply and demand.

Visual boards and 15 minute daily standup synchronization meetings also help reduce waiting for instructions delay waste in large teams where the lead also has to handle onerous reporting requirements.

Over processing waste.

Over processing means delivering more quality than the customer wants or will pay for. For eLearning that uses 3D environments or avatars, it is easy for developers to spend too much time on 3D models making the models higher fidelity than the learning objectives call for. In Agile, teams use what they call the "definition of done" for each process step to help avoid over processing waste, sometimes called "gold plating". The done definition also helps with quickly identifying abnormal conditions. instructional/LX designers and developers can sometimes add unnecessary detail in their content. Some organizations have many approvals. Some teams, when unable to go 100% Lean-Agile, inappropriately assign highly skilled people to fill out the many project status reports. Some learning experience development teams are still trying to use older technologies, such as Flash rather than HTML5 and JavaScript which cause waste. Some organizations include far too many transactions where time and effort are spent on non-value added steps which are necessary to get the needed tasks done. Although this is required waste at some levels, higher leadership can reduce or remove these obstacles to reduce waste.

Transport / Handoffs waste

Transportation waste is moving process inputs, WIP, or outputs by any means, including electronically. Although we first think poor flow due to facility layout only applies to manufacturing, in some organizations team members are located in their functional groups rather than in a cross-functional team location. Walking around may not seem like much, but when repeated often it adds up. Walking across the room or down the hall to a SME’s or team mate’s cubicle instead of having them next to you is movement waste. Another example is to hold the daily standup meeting in the work area (gemba), in front of the visual board rather than in a conference room, to avoid labor waste in coordinating, finding and walking to conference rooms. Sending emails to too many people can slow them down unnecessarily and is waste. When team members handoff work to each other, problems in communication are waste that the Kanban can help reduce.

Movement waste.

Any motion not creating value is motion waste. If team members have to go elsewhere for needed information, this is waste. With multimedia components, another example is not having a photography or video shot list and having to travel back to the system or site to take more pictures or video is movement waste. When a SME knows where a piece of information is, but instead leaves it to the learning experience development team to dig through large volumes of data instead, this is movement waste.

Inventory waste

Storage of any type of work item, information or document is inventory. Some inventory is necessary. For example, if the team has to travel to a distant site to gather photos, then it is not uncommon for the photographer to shoot many more images than end up being used. However, sorting through hundreds of images creates delay which is waste. Development team members who are new to Kanban often move 10-50 work items into their WIP column. It takes coaching to help them see that partially completed work is inventory. Pull reduces this waste. Large batch sizes increase WIP and WIP delays are waste. Exceeding WIP limits, that are designed to prevent large batches, creates delay and waste. Creating another prototype after getting positive customer feedback creates waste. Keeping months or years of old data may be required, but if not organized well the volume of files can increase time team members need to find the version they need. Lean emphasizes one-piece flow, or completing one thing before starting another to reduce inventory waste. Pulling work items rather than pushing them to the next step and having them pile up in large queues is a way to avoid inventory waste.

Defect demand waste.

Defect demand is sometimes called rework. Defect demand waste is time wasted fixing a deficiency that results in, or could result in, improper operation of the courseware, impaired learner performance, or cause rework to an existing component of the courseware. Value demand is what customers are willing to pay for. Defect demand is not something customers want to pay for. * Rework time can be measured to see how much we pay to create and then pay again to fix the courseware. Iteration demos help us get customer feedback earlier so we do not waste resources making things they do not want. Getting conflicting or erroneous information from research or SMEs is defect demand waste. Poor testing of courseware functionality and verification of its content is defect demand waste. Incorrect annotations and information to comply with accessibility requirements, such as Section 508 in U.S., cause rework waste. Annotating for accessibility too early, such as before media are created, can also create rework waste. Some buyers and producers really like so-called big up-front plans with every detail planned out. These can be the source of waste because we know the least at the beginning. When applying Lean-Agile, we have been able to convince customers and internal management that instead of planning down to every project task on the Gantt schedule, we only plan down to the iteration on the Gantt schedule, leaving some flexibility in distributed planning in later iterations as we learn more. This helps avoid much replanning later, which is rework waste.

Over production waste

Over production waste is making more than the downstream customer immediately requires. Without a daily standup, a team member in an upstream process step may still push work items without regard to the downstream WIP limits and process needs. Sending emails to everyone rather than just to the selective few who need it creates over production waste for all the recipients. This can be a hard habit to change. Having to create content for storyboarding and then again inside the courseware authoring tool creates over production waste as redundant tasks.

Unevenness waste

Imbalances in supply and demand are unevenness waste. The goal is smooth flow instead. Because spikes of demand that are higher than team capacity end up overburdening the process or series of processes, this is waste. If the customer asks for five courses at once rather than spreading them out, this can be unevenness waste. It may also be necessary waste if you cannot convince them to schedule the courseware differently. In systems thinking, having a stock or a reserve helps reduce unevenness. Rather than aiming for 100% staff utilization as traditional approaches have, instead aim to use 85%-90% of your staff capacity to have some reserve capacity for uneven demand. This is also like having savings in your personal finances. Leaders may have to buffer the teams from the customer or from management who may want to force more work through the courseware production system than it can handle. Another solution is to bring on more people. Another way to smooth the flow is to divide the work into smaller increments. For example, rather than a lesson, focus on learning objectives.

Utilization and Context Switching waste

Context Switching is time spent switching between jobs (learning curve, or memory reload). Reduce this waste by reducing interruptions for people developing the courseware during an iteration. Establish WIP limits to reduce this waste. Utilization waste also includes not effectively using the team’s or supplier’s collective talents, skills and knowledge.

Limited IT resources waste

Christoph Bauch suggested this new waste type.[54] For example, if the old courseware was created in Power Point, and now needs conversion to an online tool or to XML or JSON data formatting, the reformatting, conversions or re-entering data is waste.

Lack of system discipline waste

Bauch also suggested this new waste type.[54] An example of this would be bringing team members into the team who do not have Lean-Agile experience without providing some common leveling training. Another example might be a team not clearly specifying the iteration goal. If you use Scrum roles but don’t make them clear to the team, then this is another example of Bauch’s system discipline waste. Lean-Agile recommends setting up clear team ground rules which helps prevent Bauch’s system discipline waste.

All of these types of waste can hurt your organization, especially during fixed price contracts. The first step is awareness of what may have been unseen. Next comes paying attention and seeing these types of waste. And finally, the team looks for ways to reduce these wastes.

Mountain Goat Software has a blog that talks about avoiding the waste of “imaginary precision in estimation.” We don’t remember where we found this example, it may have been from Mountain Goat Software. Someone created a great example of this type of waste using golf. It is important to estimate a putt in inches, but golfers don’t typically spend time estimating if their next drive will be 210 yards or 210 yards and 4 inches. You may have to reinforce this in some organizations. It is okay to get less precise as a work item gets larger.

[]9.9. Working Courseware Increment

The goal of Lean-Agile development is to incrementally deliver working courseware that meets the stakeholder’s needs. This means the increment quality is ready for delivery.

Incremental gratification with each end-of-iteration demo helps theoretically prevent scope creep, but in our experience it simply reduces risk and continually builds trust (when you meet your iteration goals). Scope creep in courseware is an ever-present problem that Lean-Agile does not magically solve.

Delivering a working and tested increment of courseware every two weeks builds trust because the Lean-Agile process plans and tracks the working product increment. The delivery allows stakeholders to see and interact with the instructional content in a partial player that is working and tested, so the user interface works correctly. This direct visibility into the project status every two weeks is straightforward and concrete.

Having a two-week fixed iteration helps shorten the lead time for stakeholders or customers to get that courseware increment into their hands faster.

One week iterations may more easily fit into EVMS because of weekly updates for cost efficiency and schedule efficiency.

[]9.10. Respect for People

If you operate in an organization that tends to blame individuals first, this principle will be challenging. However, you can run your group the way you think it ought to be done and buffer them from this organizational culture.

For Lean-Agile to unlock the potential of individual people to make huge gains in productivity, you, as the leader of the new Lean-Agile effort, have to buffer people from that culture of blame as much as possible. Make clear by your actions that experiments with small failures are okay, as long as people show they are learning from the failures.

First, focus on process fixes before blaming individuals. Create an environment where Lean-Agile can flourish.

  1. {color:#000;}Is the team holding daily standups in front of the Kanban board? (Performance Feedback)

  1. {color:#000;}Does the iteration retrospective find the lessons learned in every “failure”? Blame does not foster improvement or innovation.
  1. {color:#000;}Did new people get access to the tools (virtual Kanban board, conference lines, web-based authoring tools, phones, workstation computer)?
  1. {color:#000;}Are new people introduced to the basics of Lean-Agile?
  1. {color:#000;}Is there a Lean-Agile Coach reinforcing on-the-job training during daily standups?
  1. {color:#000;}Are WIP limits being observed?
  1. {color:#000;}Do the individuals know when they are supposed to act? Are performance triggers clear?
  1. {color:#000;}Is the workflow on the Kanban board?
  1. {color:#000;}Does the current workflow need adjustment if conditions have changed?
  1. {color:#000;}Do the process step columns have an explicit definition of “done” for built-in quality?
  1. {color:#000;}Are teams using avatars to indicate who is accountable to their team mates for the specific work item cards?
  1. {color:#000;}Are teams pulling the next highest card (priority of work)?
  1. {color:#000;}Are team ground rules posted? For example, that all are expected to work?
  1. {color:#000;}Does the team know to escalate when someone is acting out of bounds?
  1. {color:#000;}Are all team members invited to and attending the iteration review/demos for regular customer feedback?
  1. {color:#000;}Does the process provide consequences to each person’s performance that is aligned with the goals of the iteration?
  1. {color:#000;}Are work items broken down into similarly sized efforts? So person A does not get stuck with an effort seven times bigger and appear to be slow?
  1. {color:#000;}Are the consequences strong enough to influence people?
  1. {color:#000;}Are work item cards updated before daily standups start?
  1. {color:#000;}Are expectations about working outside your specialty clear to everyone? For example, test cases against acceptance criteria checklists.
  1. {color:#000;}Is the team work pace sustainable indefinitely? Does it lead towards fatigue and poor decisions in the short term and burnout over time?
  1. {color:#000;}Are the people who do the work expected to work out the improvements? Yes is better.
  1. {color:#000;}Are instructional/LX designers open to feedback from team members about how well their intent is communicated?

After assessing the process, then look at the person.

  1. {color:#000;}Does the person know how to do the job?

  1. {color:#000;}What is the movie like, versus the snapshot? If one observation is like a snapshot, then many observations over time are like a movie.
  1. {color:#000;}How does the person feel about the situation?

[]9.11. Small Batch Training at the LO Level

Lean causes you to reduce your batch sizes. Rather than telling the customer we’re working on lesson 2, tell them we’re working on courseware increment #2, during iteration #2. Provide a scope commitment to them for that iteration, so they know that this courseware increment includes learning objectives (LO) 3.1, 3.2, and 4.1 through 4.5.

During the iteration, focus the work on a single LO at a time. Storyboard the one LO rather than the entire lesson. Then get the interactivity done for it and the media created for it. Perform the QA content checks for that LO. Define ‘done’ for the LO as ready to send to the LMS or learners. Now ideally you don’t have to touch this LO again. In the real world, the SME or customer always says, “Can you also change this and that?” So, you will have some degree of changes (rework discovery rate). Estimate historical rework rates, for example 10-20 percent, depending on the customer stakeholders involved with review. Monitor rework and talk to the buyer/customer if it gets above your planned amount.

Smaller batches also mean less scope, so fewer people are trying to coordinate with you at the same time, which reduces interruptions, and therefore reduces the waste of context switching.

Smaller batch sizes also means more even flow through your development pipeline, that customers see and inspect increments every two weeks. This builds trust between the customer and your team. It also helps your team get earlier feedback before they are done with the entire lesson. It costs less to do work once than twice, so earlier feedback reduces risk and helps your bottom line.

[]9.12. Standard Work in the Form of Checklists

Lean principles include the identification and codification of standard work. Standard work is not a replacement for skill and knowledge, its purpose is to enable skill and knowledge to be applied consistently and effectively.

Efficient development of high-quality courseware and supporting player software is not a touchy feely, unmanageable process. It is a creative process, but one that benefits from thoughtful application of defined principles, practices, methods and techniques. Standard work checklists developed by the Lean-Agile teams are our systematic development practices.

Lean author Jami Flinchbaugh points out [55] in The Checklist Manifesto: How to Get Things Right, that author Atul Gawande explores applying standard work to the complex field of medical work. If standard work using checklists can improve the work of specialists like surgeons and pilots, it can also work in learning experience development. Checklists can be the main way we apply standard work.

Atul Gawande’s book is worth reading. He writes well about how checklists can help. We think checklists are a domain crossing idea that can help in learning experience development too. Gawande points out that checklists are not formulas, but reminders to help performers be systematic to reliably perform.

Gawande clarifies that checklists are “not comprehensive how-to guides,” but are only meant to “buttress the skills of expert professionals.”

So why don’t we already have checklists in use in all complex work? Atul Gawande talks about airline pilots and medical doctors using checklists.

We don’t like checklists.

— Atul Gawande

So like pilots and surgeons, those of us in the learning and development discipline use checklists as simple tools designed to buttress the skills of expert instructional/LX designers, developers, and media professionals.

[]9.13. Make Defect Demand Visible

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<>{color:#000;background:rgba(0, 0, 0, 0);}. If your organization’s definition of “defect” makes this too hard, then think of it as rework demand.|

Because we get to pay for making defects and then pay again to fix them, we use the Kanban board to make defect demand visible using a different color of sticky note. We use orange stickies to differentiate defect demand from value demand (requirements).

The different color card for defects becomes explicit and visual input to iteration planning, prioritizing and estimating the effort involved to fix defect demand.

Unlike a bug tracking program to track courseware defects, the Kanban board makes the amount of defects visible to all. When you zoom out on virtual Kanban tools or step back on whiteboard Kanban boards, the number of orange sticky notes tells a story. It also gives insights into how differently both parties understood the desired solution.

You can quickly see the visual proportion of orange defect stickies for demand as compared to the colors used for value demand. Some management groups will also ask for a pie chart with defect/rework percentage and value demand percentage.

In some virtual Kanban systems, we use a specific Kanban board lane (a horizontal swim lane like in a swimming pool) for defect tracking if the process steps (vertical columns on the Kanban board) are different from the original task. In those virtual Kanbans that don’t allow lanes, the color still gives a visual indicator of the demand types.

Noting the defect arrival rate (the rate at which defects are discovered) can provide insight into the likelihood of making the target dates.

Defects can also be an opportunity to learn how to improve the process and prevent bigger defects.

Don’t treat defects as negative things to be avoided or people may be less likely to disclose them and may waste too much time on certain tasks in an attempt to avoid mistakes. If they cover their mistakes earlier in the process and you don’t catch them until later, it will cost more to fix them. Better to find out earlier and fix it before more effort is spent. Consider defects as part of the process and fix them rather than focusing on where blame should be assigned. You’ll deliver faster this way. If defect counts drop then lead time drops too.

We also add defect cards when stakeholders email a comment. This gets it into the visual management system and out of the hidden email system. Virtual Kanban cards also provide the audit trail of when the card entered and left each process work step in the event an audit is necessary.

The virtual Kanban board helps us gather the metrics on rework rate of discovery and rework resolution.

This tracking of defect demand is especially important for customers who have far more than expected rework. It provides data with which you can address the situation with the customer stakeholders and your leadership.

[]9.14. Customer Collaboration for Earlier Feedback

One of the more valuable aspects of the Lean-Agile approach is the increased feedback from customers earlier in the process. Every two weeks, we have a meeting with the customer that demonstrates the courseware increment. Every two weeks, the development team hears concerns by the stakeholders in the review/demo meeting. That feedback can be applied to the next two-week iteration. We try to get all the decision makers involved in the demos every two weeks.

Some buyers/customers initially worry that iteration reviews add additional cycles of time to their reviewers. Then after going through the process a couple of times, they end up liking it. We have also seen a reduction in the number of end-of-development change requests (CRs) because the buyer’s stakeholders have been able to voice their issues early, and the development team fixed it early.

This increased collaboration with the customer stakeholders helps the customer trust that the courseware is coming along, and helps the development team see what the customer likes and what they want changed. We have experienced discussions where the SMEs disagree and come to consensus during the demo. This is much faster than arranging an additional review by an approved referee and waiting for review comments. The entire group of buyer stakeholders is present, including the customer leadership, and this collaboration is witnessed by our development team. It is valuable for helping the development team see what is important to the customer.

For example, how the customer wants to handle accessibility (Section 508 in U.S. Code) for eLearning courseware may come up in the first iteration, and that feedback applies to all future iterations. This lowers risk. If we had done 508 a particular way for more of the content before later discovering the customer wanted a change, we would have more rework, and the buyer would at least have a delayed schedule and likely increased costs too.

The increased collaboration helps the development team adjust to what the customer likes. We have found the teams look forward to these collaboration meetings because they hear what people think of what they have created. It is especially motivating for the team when the customer stakeholders provide “I like that” comments.

Inviting all the team also helps expose team members to more customer interactions, growing our talent for future projects.

[]9.15. Continuous Improvement – Kaizen

Kaizen, Japanese for improvement. When used in the business sense and applied to the workplace, kaizen refers to activities that continuously improve all functions and involve all employees from the CEO to the line workers. It also applies to processes, such as purchasing and logistics, that cross organizational boundaries into the supply chain. It has been applied in healthcare, psychotherapy, life-coaching, government, banking, and other industries.[56]

Although many organizations talk about Kaizen, continuous improvement, when you use Lean-Agile, you get an environment that more directly supports Kaizen with the iteration retrospective meeting. The visual Kanban board lets the entire team see that status of all the work in the project. The daily standups provide frequent opportunities to see the board as it changes.

When the entire team can see all the data that normally is only available to the person with the MS Project file or spreadsheet with all the status data, then the brains of all on the team can engage towards improving specific problems. This facilitates ideas from everyone, making better improvements regularly.

With every iteration, the team can put their heads together to get a little better and find more impediments to remove or Lean wastes to identify and eliminate.

We have had to get some teams past the idea that this is only for some documented “lessons learned” process and that they can actually make their own work easier by doing the retrospective each iteration. Time boxes on this meeting keep the duration manageable.

[]9.16. Planning Still Happens in Lean-Agile

Lean-Agile does not promote zero planning. Planning is conducted in Lean-Agile. Where it differs is that it has a smaller part of planning up-front and also includes planning efforts distributed to each iteration.

[]9.16.1. Planning Versus a Plan

“Prepare for the unknown by studying how others in the past have coped with the unforeseeable and the unpredictable.” [57]

— General George S. Patton

Planning does not dominate the project so completely up-front as in plan-driven all up-front waterfall approaches.

Lean-Agile means something different by planning than what some traditional project management approaches have come to mean. Traditionally, there has been a heavy emphasis on planning meaning the plan, as in an artifact. If the operational environment is certain and predictable, then that can work. However if the operational environment is complex, unpredictable and higher tempo, like warfare, then what our military customers do can also be instructive for business teams.

[]9.16.2. Planning Prepares the Mind for the Coming Challenges

As military history bears out repeatedly, and is taught to military officers, planning adds value, but the plan itself has very little value.

The U.S. Army has a saying that “No plan survives first contact with the enemy.” They say this not to devalue planning, but only to acknowledge that what will actually need to happen may be different from the earlier plan. We mention this to highlight that planning is dependent on the information available at the time. Agile allows planning to be adjusted to the new realities that exist at the time of execution that may be quite different from the original envisioning of the project up-front as more information has become available over time. When using earned value, think of this as rolling wave planning.

In military warfare and in Lean-Agile, planning is discovering the factors involved in the circumstances and the systems structures (systems thinking) that create dependencies and feedback loops which constrain our decisions. So planning is more about adding insight and wisdom to team/organization decision making than simply having an artifact called a “plan”.

Planning makes us better prepared to respond to the changing situational context. Planning helps get us into the mindset to see opportunities.

Where possible, it is better to have the same people who planned (and thus built the insights) execute the plan, rather than having different people analyze/plan and execute to avoid the loss of their mental situational model when the planners move to other assignments when execution begins. The plan artifacts that remain are often many large documents people don’t want to read and that don’t keep up with the execution anyway. New staff charged with execution must replan (rework is waste) for their own situational awareness, so they are ready to respond to changing conditions.

Reading someone else’s plan document does not provide team members the contextual awareness that doing the thinking, making findings and forming their own conclusions provides.

The goal of Lean-Agile planning is not to create a large plan document, but to get ready to execute well. In less complex, predictable environments, a deterministic plan may succeed. In a complex adaptive system, discovering what is happening and how to adapt is the key.

“In preparing for battle I have always found that plans are useless, but planning is indispensable.” [58]

— General Dwight D. Eisenhower

Lean-Agile planning puts the value on what is learned and discovered while planning more than putting value in the artifact itself. Rather than a long written plan, the product backlog and the iteration backlog form the “plan” of intended work items for the project or specific iteration.

Rather than predicting what will happen and then following the plan exactly, we’re focused on gaining the mental model of the environment. The U.S. military also works in complex environments, and in their lexicon they call this “visualizing the battlespace” to help clear the fog of war. So in Lean-Agile as we plan and gain the mental model of the environment, it helps us clear the fog of complexity during the quickly-changing execution.

[]9.16.3. Project/Program Definition Planning

Project/Program definition planning still includes stating the aim of the Project/Program and identifying Project/Program objectives.

The work breakdown structure (WBS) is detailed only down to the iterations, delaying planning the specifics of what occurs in each iteration until closer to the iteration so that all involved have the latest information.

Resource requirements are identified for the people and tools needed.

Traditional up-front deliverable sequencing, resource scheduling and risk management is mostly distributed to iteration planning meetings.

Roles and responsibilities are discussed later in the book.

Figure 41. Planning Basics

Lean goes so far as to say that too much time spent planning everything up front is a form of Lean waste because the situation emerges and changes as more is known.

The following figure depicts the differences in when the planning occurs. Note that the same 100 “units” of planning effort still occur, but they are spread or distributed across the iteration planning meetings rather than all up front. Like rolling wave planning the planning has more value when the effort is delayed until more is known, leading to less planning waste.

Figure 42. Comparison of Traditional ADDIE Planning and Lean-Agile Planning

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. For more details on planning, see Conduct High-Level Planning Tasks skill statements in Appendix A.|

[]9.16.4. Product Planning

During project/program planning, a Lean-Agile team conducts planning to wrap their head around what is involved overall. The backlog is the current ‘plan’ given what we know now. The development team may estimate the work effort in the training product requirements backlog items.

[]9.16.5. Iteration Adaptive Planning (recurring, like rolling wave planning)

Lean-Agile plans the high level up-front and allows room to plan the details in the iterations where the information is more up-to-date.

During iteration planning, the iteration backlog plan is pulled from the product backlog. The iteration backlog plan is visible to the entire development team on the Kanban board, and it is prioritized so that the team knows which work items they should pull next.

Iteration planning also helps teams adapt to emerging circumstances based on the parameters of the operating environment about which our planning provided so much discovery and insight. The Scrum method builds-in a closed-loop control process to get feedback (+, -, change) fed into each iteration planning meeting, so the team can adapt for the next iteration in striving for perfection.

[]9.16.6. Daily Adaptive Planning (recurring)

Because of complexity, the plan is then adjusted in the daily synchronization meeting when the team applies PDCA.

Whether you call them daily synchronization, daily scrum or daily standup meetings, the team adjusts and adapts based on what is actually happening. Having gained a mental model of the environment, they can recognize the impact of new and unexpected risks and opportunities as they emerge and decide the handling actions in a daily frequency rather than the monthly frequency of traditional sequential waterfall ADDIE. Each day in Lean-Agile, the team looks together at the situation on a visual Kanban board, seeing the current status and visual cards representing both planned (predicted) and newly emerged (unpredicted) risks and opportunities. This allows them to collaboratively decided what steps to take for the day. As in a coach’s soccer game planning, opportunities come up fast for shots on the goal, and the opportunity is fleeting. The ‘plan’ is not the goal. Being ready to score is the goal. Daily standup planning is the place to see and exploit those game changing opportunities.

For anyone you run into who thinks or has heard that Lean-Agile does not include rigorous plans, we can assure them it has rigorous and distributed planning and that the planning is the foundation of decision making. The emphasis differs and is less on the artifacts themselves and more on what’s in between the ears of those planning. This includes discovering and balancing the constraining lay-of-the-land factors involved in the circumstances and the systems structures (stocks and flows) that create dependencies and feedback loops so that we make better decisions during execution.

[]10. Adaptations to Agile for Training Development[]

[]10.1. Lean-Agile Design–Still Experimenting–Stay Tuned …

Lean-Agile principles apply well to development, and we have experienced much success using it in development efforts. Yet for designers, applying Lean-Agile principles is still a work in process.

In the software domain, sometimes many of the important design decisions have been pre-determined by choices of operating system servers, programming language, database, etc. In the learning and development domain, instructional/LX design happens on every single project.

We have found it more challenging to apply Lean-Agile to the design portion of ADDIE. Historically, training projects started with a course design document (CDD). Depending on the organization and their process orientation, we can build the CDD with revision one and two, or draft and final, but our experiments with Lean-Agile have not yet yielded a great solution for design that has helped us convince buyer/customers to abandon big up-front designs of the entire course. We have tried to persuade customers that desire a CDD to allow it to be course-level only design—meaning mostly scope and sequencing of LOs, and assessment strategies with generalized instructional/LX design strategies—at this point in the process and leaving all the lesson-level design—meaning specific instructional/LX design strategies to implement the enabling learning objectives—to be distributed across the iterations, with each iteration having some design effort and some development effort. However, we have not convinced all customers to adopt a more Lean-Agile design approach. Some want an up-front big picture or blueprint of the course. Some want so much detail in their up-front CDD that it is almost a storyboard, just shy of screen-level/page-level details.

Recently, we have also noticed an emergent and promising possibility for making learning experience design more visual (Lean), flexible and easier to communicate with stakeholders for earlier feedback loops (Agile). It is called the Learning Environment Modeling Language (LEML).[59] LEML also seems to offer a usable path to a minimum viable design. For us, applying Lean-Agile principles to design efforts and getting customers to agree is still a work in process.

Traditionally, design is messy and hard to describe already.

For some, the greatest diagram of the design process was made by Damien Newman and is shown below. His diagram helps other people get immediately the initial messiness and nonlinearity of design.

Figure 43. The Process of Design from a great height by Damien Newman [60]

Eric Ries talks in detail in his book Lean Startup about how early on we should iterate experiments to learn what the customer will pay for. Designing learning experiences can use that approach too.

We think Damien Newman’s design diagram captures well the degree of effort trying to test our set of customer value predictions in the research and concept sections of his design process diagram. Often, the discovery of what the customer really values for learning is the largest constraint initially. Sometimes this discovery constraint creates a traffic jam in the project execution until both we and the customer collaboratively reach clarity on their value demand—what they most want and will pay for. Iterations and feedback help. We will talk more about constraints and their impact in the section about the Theory of Constraints. The asserting, experimenting and learning process applies to designing learning experiences both in mature companies and in startups.

Our experiments with Lean-Agile for design indicate a need to change your relationship with buyers/customers. Moving to Lean-Agile means a big mental change and different process steps for both customers and your development team. Rather than thinking of only delivering perfect, detailed, high quality designs built over longer design calendar time durations, work out a shorter design cycle with customers at the beginning of the project.

Unlike the functionality of eLearning, learning simulations or mobile learning, the design effort is not as easily decomposed into small similarly sized chunks. The design process characterized by Newman’s figure shows that the work breaks down to research/discovery with much initial uncertainty and then insights and concept/strategy gain clarity, which is followed by focus for the design.

The instructional/LX design work breakdown may include analysis discoveries first, then learning experience discussions. Then if creating a CDD, perhaps, go to scope priorities and sequencing consensus on proposed learning objectives (LOs) and your proposed assessment strategies. Next, conduct instructional strategy collaboration meetings with your stakeholders and subject matter experts. Then, write it all up in a CDD to preserve what emerged from all the back and forth.

Trial and error indicates one alternative that does not work well is surprising the stakeholders and SMEs by disappearing for long periods of design time and then delivering an entire design with little or no customer input. Customers often want to feel a part of the design effort, and leaving them out of design does not play to their desire to help shape the outcome of the training intervention.

A too frequent reality is that customers may not be completely sure of what they want from the start due to changing conditions and stakeholders. Customers gain a clearer understanding as we propose solution ideas and rough sketches and verbally paint pictures. Or better, demonstrate low-fidelity prototypes. Emerging customer desires are why the Lean-Agile adaptive-versus-predictive approach has succeeded in the customer’s transition to clarity. The customer’s environment not static and external demands on your buyer/customer stakeholders may change during the project causing their needs to change and what they want from the learning experiences to change too. So theoretically, if there are more small cycles of collaborative design exchanges of ideas, then changes are easier to accommodate. Customer availability impacts more frequent cycles in real projects. Set expectations that the Lean-Agile Courseware approach requires the customer’s engagement beyond just the kickoff meeting and the final review.

When customers are not paying for learning experiences, they often want courseware yesterday. When customers are paying for learning experiences, they often want a fixed scope up front before they start releasing more money. Try describing Lean-Agile as the children’s game red-light, green-light. The customer gets to say red-light and stop and see an increment and inspect it before saying green-light to release more money to begin another iteration. Although a new approach, they actually have and feel more control and less risk because they too get more feedback earlier in the project lifecycle.

Customers also expect a coherent courseware vision for effective learning experiences. A coherent vision benefits from some up-front identification of scope boundaries, design constraints, assumptions and desired outcomes of the intervention.

As a side note, working for a smaller training group as a cost center inside an organization making training interventions for organizational members may not require as formalized of a design approach as business-to-business and government contracting. In some circumstances CDDs can be seen as the Lean waste of overproduction, also called gold plating.

Agile software domain practitioners call their equivalent to CDDs a Big Design Up Front or what in the ADDIE domain we often call the Design Phase. Traditional recipes for design typically call for the ingredients of research, ideation and time to craft a carefully synthesized long-term vision for the courseware product. In contrast, Lean-Agile development iterations tend to focus on the next two weeks.

The design effort is typically a scheduled time period where designers can take the project requirements and ideate, often at length, over how the solution will align with and solve the customer’s problems from the training needs analysis. Instructional/LX designers will explore optimal ways of writing LOs, strategies for assessing LOs, and then various instructional strategies to achieve each LO.

Designers new to Lean-Agile may initially protest that the fast-as-possible principle does not give them enough time to think about the solution, and some may say it is not possible to turn around a design in shorter periods. If you’ve set the conditions with the Customer and internal management to expect multiple smaller design cycles rather than one giant step for design-kind, then it becomes key to focus hesitant designers on only one design cycle’s scope of design work.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. If you can get the customer to agree that working courseware increments replace document artifacts as the primary evidence of progress, in the contract if necessary, it can go a long way to helping manage expectations. If you have to use a CDD, an important distinction is the level of design detail. Our experiments indicate that it helps if you set expectations contrasting the high-level design (CDD) being minimally detailed—or dropping design up-front and distributing it throughout the iterations—against the detailed designs, called storyboards.|

Customers that are familiar with ADDIE and traditional approaches to training and learning interventions may have expectations of comprehensive, large-batch CDDs up front, rather than a more Lean-Agile skeletal design framework to be fleshed out in detail during the smaller-batch design cycles that are distributed throughout the project in iterations.

The key mindset obstacle is historically large batch designing and storyboarding of the entire course before development ever starts. Teach customers that small batch courseware increments let them see a completed, ready-for-learners increment earlier, which helps them decide how closely your idea of a learning intervention solution matches their sense of the training need, which may be changing as they gain clarity on what is wanted. Help the customer understand that large batch storyboards (the entire course) are still subject to misunderstandings as humans encode one person’s mental model to words and another person decodes the words and attempts to form their own mental model. Like the children’s game of passing a message down a line of multiple children, what comes out the end is frequently not what went in. Seeing a small chunk “done” makes it harder to mismatch the mental models because the “done” increment is in its final form for learners. Small batches mitigate the risk that occurs when customers are waiting on a large-batch storyboard. Small-batch increments improve clarity between customers/buyers and developers earlier in the process.

Here we often explain that a model is a representation of our current understanding. It can help to make use of low-fidelity visual models—any visual representation of our current understanding—to provide focal points for discussion and to help build shared understanding.

Storyboard design, if required, can be addressed a little in this iteration and a little more in the next iteration. CDDs are typically frozen upon approval (also called configuration managed) and are more difficult to change, often with contractual requirements in play. Instructional/LX designers who are used to design-once-and-cast-into-concrete may relax somewhat when they realize they don’t have to design the entire storyboard at one time, perfectly. Lean calls this just-in-time design.

|={color:#000;background:rgba(0, 0, 0, 0);}. Tip|<{color:#000;background:rgba(0, 0, 0, 0);}. If designers get too far ahead of the development team in storyboarding (detailed design), they may introduce Lean waste of context switching after they’ve moved on from that LO, as well as if the customer priorities shift or other changes occur. In Lean-Agile, we’re optimizing the entire team, not the individual designer’s process. Also with the Theory of Constraints, the instructional/LX designer, and game designer if that applies, become(s) the drum beat to which all the other process steps should march in concert because instructional/LX designers and game designers tend to be the primary constraint.|

Perhaps you can convince your customer at the start to go as light as possible (skeletal) on the detail of the high-level design/CDD or convince them to push the up-front design effort into the iterations. But in some cases, ADDIE-aware customer leadership experienced in waterfall may not agree, so be as Lean-Agile as you can given your context. One way to propose this for fixed price work is, “You get X amount of work effort (size points, hours, etc.) and any change is okay as long as you swap out an equal amount of lower priority work to replace it with the newer change.”

|={color:#000;background:rgba(0, 0, 0, 0);}. Tip|<{color:#000;background:rgba(0, 0, 0, 0);}. Fixed-price contracts generally do not support changes inherent in Lean-Agile as well as Time and Materials contracts.|

Another way to operate in Agile is to use what the software domain calls the Minimum Viable Product. This minimum idea is anathema to many designers. They have designed robust learning experiences that demonstrate their years of training and experience. The idea of releasing an experience that is streamlined or “good enough” feels like low-quality. You can coach your customers and designers into an experimenter mindset. Each iteration is an experiment to test our hypothesis that our design increment will meet their training need or problem, also called value demand in Lean, and is fit to purpose. Most of designer’s professional training is to do the “big picture” design in one large batch up front rather than a Lean-Agile series of lighter weight, smaller batch, cycles of design where the customer gets to inspect and adapt, and everyone involved gets earlier feedback that validates the potential of that design chunk.

Another approach is to build on the familiar. Instructional/LX designers and many customers are already familiar with the idea of pilot testing courseware, so iterative design can be described as a series of mini-pilots for the same purpose: to gather feedback about the degree to which that chunk is fit to purpose. This is the PDCA or Deming Cycle.

Use a new minimum viable design question, “What is the least amount of courseware product we can design to determine whether or not this approach is valid?”

This question uses the Minimum Viable Product as a form of customer research. Who decides what is minimally viable varies (Instructional/LX Designer? Customer? Lean-Agile Product Owner? Contract provisions?). Collaboration is key. By including the instructional/LX designer in this decision, you can increase their comfort level with releasing what they may feel is an incomplete courseware product.

Another challenge in applying Lean-Agile to design is that Lean-Agile encourages a steady cadence and flow.

Agile iterations are planned to create a steady development velocity, but designers don’t always benefit from the same uniformity of workload.[61]

— Cennydd Bowles

A steady cadence can be difficult for designers because their design effort is so often not initially a smooth linear flow. Damien Newman’s process of design drawing is superb for helping communicate this unevenness of research, concept and design. It is challenging to account for the initial research and the associated design uncertainty with Lean-Agile. Iteration zero may help. Michael Allen’s SAM method advocates design iterations with multiple prototypes—​which can work well when the customer agrees to multiple prototypes. Interactive paper prototyping is quick and inexpensive—​to the degree you’re able to keep it constrained to low fidelity—​and it facilitates Lean-Agile more easily than static CDD documents and static storyboards.

Fortunately, we can learn from other fields. Filmmakers operate in a similarly agile fashion, filming scenes in an order dictated purely by logistics. To ensure vision, coherence, and narrative continuity they employ specialists: directors and script supervisors.[61]

— Cennydd Bowles

Learning simulations and serious games also chain together player experiences into entire player journeys, often called missions and campaigns to achieve learning outcomes. CDDs and static storyboards don’t reflect well the dynamic experience of technology components for learning. In courseware, the CDD behaves like a movie script, attempting to chain together work items (or user stories for simulations and games due to the software developers on those projects already being familiar with terms for work items from the software domain) into whole journeys that can be tested for cohesiveness. This approach can reveal much about a learner’s overall experience and can expose gaps missed with the heads-down iteration focus.

Another factor that recently impacts instructional/LX design is the multiplicity of screen sizes. Historically, the page was the atomic unit of courseware, but no more. HTML5 responsive design for mobile-first designs automatically reflows page blocks into wide-screen multi-column formats for PC screens and large tablets and into a thin-screen single-column formats for mobile devices. Web and mobile app users now expect this behavior. When these users become learners, they expect it in learning apps and courseware too. So instructional/LX designers should move beyond the page construct as the organizing principle and move to learning experiences instead.

One of the L&D industry thought leaders, Cathy Moore, developed an alternative instructional/LX design model called Action Mapping [62] for designing learning experiences may better fit Lean-Agile than CDDs. Using Cathy’s Action Mapping for design iterations means we can cycle through the goal, to the actions, to the activities and the content. Action Mapping is better suited establishing the courseware vision and iterative design than a Big Design Up Front approach with CDDs. Cathy’s approach supports minimally detailed planning up front and still provides a coherent big picture skeleton upon which to flesh out the details during the courseware development iterations. Her approach also supports minimum viable product courseware by filtering out unnecessary content. In our experience, CDDs tend to encourage extensive documentation as the primary measure of progress rather than working courseware increments.

Interactive working prototypes are also better than static document-centric design increments. Game and simulation prototypes have to be play tested to see the dynamics at work, creating the overall effect and player/learner experience, and to validate it against player experience goals and learning objectives.

More and more learning experiences are enabled by software, which is why we borrow relevant ideas and patterns from the software domain for courseware product development. So we can learn from user experience (UX) designers in the software domain too. UX designers use low-fidelity sketches and paper prototypes as ideal fast and cheap experiments for sharing design ideas and exploring discussion points. Fail-fast-and-learn patterns still seem infrequent in instructional/LX design practice in corporate and government training. Some instructional/LX designers primarily see the world from the left-brain, text-oriented design depictions and models. We can reduce misunderstanding waste by tapping more into the right-brain, drawing/visual-oriented design depictions and models. The encoding of the designer’s mental model into the low-fidelity format of text has to be manually decoded by the readers/stakeholders/customers, and the mental model that they form is often mismatched from the designer’s mental model, while both parties may think we’re all aligned. This increases risk and delays its discovery and resolution until further into the product development lifecycle, costing more for the customer (if variable priced) or for your organization (if fixed priced). Drawing/visual-oriented design depictions and models encode the designer’s concepts in a way that takes less effort by the viewer to review and often can lead to a better grasp of gaps or differences in mental models and provides an improved focal point for discussions.

A picture is worth a thousand words.

— Unknown

User experience designers also have to iterate their design work.

By sketching quickly and often, UX designers rapidly iterate on their work while building up a paper library of components, buttons, navigation elements, drop downs and so on.[61]

— Cennydd Bowles

Instructional/LX designers and game/simulation designer teams can also apply this successful pattern building up their own paper library of UX design components needed for courseware, games and simulations.

Showing solution components early and often can feel uncomfortable initially and it helps become Lean-Agile.

In summary, how far you get with Lean-Agile design has more to do with persuasion and negotiation skills than with Lean-Agile techniques. Adjust for your negotiated context, applying as much Lean-Agile as possible within your constraints.

[]10.2. Kanban is the Most Flexible Workflow Tool Yet

Back in the old days, workflow tools required high licensing costs, database configuration and special tools to design workflows. All of this was painful and not as helpful as promised. Complex setup and configuration and time consuming change processes meant these tools became less used over time. Complexity killed these tools off.

With Kanban, the workflow can be set up simply.

  1. {color:#000;}Assume: The initial workflow is clearly understood.

  1. {color:#000;}Initial configuration:

–Dry erase marker (physical board)

–Press “New Lane” (virtual board)

  1. {color:#000;}Change the process configuration (possibly each iteration as lessons are learned):

–Dry eraser and marker (physical board)

–Press “Edit Lane” or “Move Lane” or similar words depending on the supplier (virtual board)

We learn a lot with each new customer. We often have to alter our workflow to better fit the changed circumstances. Changing a whiteboard is simple. Changing a virtual whiteboard is only slightly harder.

We have easily modified the process for various training delivery methods. In a five-iteration project, the workflow was changed four or five times. The changes were tweaks, but easy to implement. The instructor-led training course includes steps like instructor guides that we do not make for eLearning or all mobile learning.

The workflow is locally scoped to the Agile team and the team is empowered to change the workflow steps if needed. The teams have recommended changes and because the fix takes 1-2 minutes, they continue to suggest changes. On older, more rigid workflow tools, the fixes took much longer and involved approvals adding wastes of delays and transaction costs.

To apply the Theory of Constraints, Lean and measure wait time, we have “done” columns after each WIP column. The time the cards spend in the done column is waste so the team tries to minimize time spent there. The pull system and WIP limits help with this too. The definition of done also helps the team have more consistent quality built-in rather than added at the end. Done columns also help measure touch time versus wait time. Not using done columns leaves waiting time unseen and not improved.

[]10.3. Kanban as a Process Map

Making your process visible helps everyone know what is expected. Process mapping is about knowing what the current process is so we can make improvements. Rather than a process map or value stream diagram in policy documents that no one ever reads, putting the process steps on the Kanban board makes the process steps visible to the development team and to higher ups who may want to shave time from the process.

Once the process is visible on the Kanban board, then the team can see the state of the project easily. Like the traffic helicopter gets high above the streets to see the state of the commuting process, the Kanban board puts everyone on the team in the position of seeing the state of the entire learning experience development process. Seeing it all at once makes it easier to spot bottlenecks or traffic jams. Seeing problems is the first step to improving them.

The Kanban board makes the process steps explicit which also helps when someone resigns, is replaced, or if the team size increases and new people are added. The new person sees the entire process on the board without having to experience it by OJT before understanding it, often much later.

In times of growth, when scaling by adding more teams, the visual process sequence also helps new people get aboard more quickly because they will see the process on the board during the daily synchronization meetings. On-boarding training is supposed to cover policies, etc., but they often don’t retain all that in our experience. The Kanban board is repeated frequently and helps burn in the current process. This is especially important during spikes of super fast growth where many teams are added nearly simultaneously.

Add a “Definition of Done” card to the top of each “done” column so the built-in quality criteria are explicit and they know they have met it before moving a work item card to the done column on the Kanban board. Reinforce with the team members that they are committing to the entire team that they met that definition of done before moving the card to the “done” column. This accountability to each other helps the team perform better.

[]10.4. What to Track on your Kanban Board

The first time we tried to apply Agile Kanban to a project, we went overboard and added a Kanban visual card (sticky note) for everything. We had too many cards. It frustrated our first team.

Of course, this was not the best idea. Over time and with many instructional/LX designers providing input, we have currently settled on tracking the following things for courseware with significant software code.

  1. {color:#000;}Deliverables

–CDD (if required)


–Courseware Package or Instructor Support Package

  1. {color:#000;}Lesson Level Work Items

–Lesson 1

–Lesson 2

–Lesson 3, etc.

  1. {color:#000;}Learning Objective (LO) Component Level Work Items

–LO 1.1 text

–LO 1.1 media

–LO 1.1 test items

–LO 1.2 text

–LO 1.2 media

–LO 1.3 test items

–et cetera

  1. {color:#000;}Task Level Work Items

–Risk mitigation tasks

–Quality tasks

  1. {color:#000;}Change requests (CRs) / Defects

For projects with significant software code, like simulations and game engine work, user stories and epics are more appropriate work items because the practitioners are more often software developers than training professionals.

[]10.5. Requirements as Work Items

In commercial corporate training and in training for learning and development groups, we have not found the term “requirements” to be commonly used by training groups or HR groups as it is normal for quality engineers, systems engineers and software engineers. More common are training needs and learning objectives. However, in engineering organizations “requirements” are part of the normal lexicon and processes, and in our communications with these groups, we find it easier to use their vocabulary in persuasive endeavors.

A requirement is a singular documented physical and functional need that a particular design, product or process must be able to perform.[63]

ISO 9001 addresses learning customer requirements, meeting them, and measuring how well you met them. Many systems engineering and software engineering projects start with requirements elicitation or generation and a product breakdown structure. Agile software teams tend to use the terms “epics,” “user stories” and “tasks” to indicate particular levels of work in the project.

We have found, however, that instructional/LX designers and media staff are more often unaware of the software terms, and these terms may form obstacles when adopting Lean-Agile in training organizations. If you have the need for quickly expanding teams or have people pulled from projects to support other groups, then these terminology obstacles can have even more impact.

In our experience, the more generic and less abstract term, “work items,” goes over just as well with training professionals.

Much of the current Agile literature focuses on software product development and describes user stories as the way of expressing requirements, or even collections of user stories forming a larger unit they call epics. The software development community uses this vocabulary for Lean-Agile because it helps scope the work for a completely new product breakdown structure on each project as we continue to reduce batch size for more even flow—a Lean principle.

Consequently, for training teams where we have introduced Lean-Agile, we have called user stories plain old “work items.” Work items are more generic and not as software-centric. For training teams supporting software products, it may cause less confusion for the software developers to stick with epics and user stories. But if your shop is not tied to software development, you may find that the language from the software domain is not comfortable for training professionals because of their long exposure to the terminology of training.

You might ask, “Why not just say tasks?” The answer is that a task is just one level of granularity of a work item. We are trying to make the development work in process visible and so need Kanban cards for intermediate work products at all the levels of the courseware product breakdown structure.

As to the courseware product breakdown structure, the training community already has many years using the following product breakdown structure, even if they don’t call it that:

  1. {color:#000;}Curriculum

  1. {color:#000;}Courses
  1. {color:#000;}Modules (sometimes)
  1. {color:#000;}Lessons
  1. {color:#000;}Learning Objectives
  1. {color:#000;}Key learning points or activities

This pattern of specific terms to represent an increasingly smaller scope of work as you go down the list of terms is not new, and it has worked for many years for the purpose of identifying what granularity of scope is meant.

In adapting Lean-Agile to training groups, we followed the pattern of “If it ain’t broke, don’t fix it” and consequently adopted the preceding and historically most commonly used courseware product breakdown structure.

For pure training teams, work items will continue the breakdown with the following:

  1. {color:#000;}Curriculum (if applicable)

  1. {color:#000;}Course (if applicable)
  1. {color:#000;}Module (if applicable)
  1. {color:#000;}Learning Objectives (LOs)
  1. {color:#000;}LO Text components
  1. {color:#000;}LO Media components
  1. {color:#000;}LO Test Item components
  1. {color:#000;}Tasks
  1. {color:#000;}Other requirements (contractual, UI, player functionality, etc.)

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. If you use an XML-based authoring tool that includes semantic elements for your work breakdown, you can tie these two together by using the same terms that your XML schema uses. Again, it is worth repeating, Lean-Agile asks us to make the development work in process visible, and so we need Kanban cards for intermediate work products at all the levels of the courseware product breakdown structure.|

[]10.6. Build the Courseware Backlog

For training projects that use a Courseware Design Document (CDD), this lays out the backlog for the project. The courseware gets a Kanban card. Each lesson gets a Kanban card in the backlog. Each learning objective gets a Kanban card in the backlog. We have found that adding the following additional cards for each LO helps monitor the work better.

  1. {color:#000;}LO X text component

  1. {color:#000;}LO X media component
  1. {color:#000;}LO X test item

It can be helpful to think of parent-child relationships in courseware structures. An LO is a child of a lesson. A lesson is a child of a course.

As you complete the child work item components of the LO, then the LO card moves across the Kanban board. As you complete the LOs, the lesson card moves across the board.

[]10.7. The Courseware Initial Plan

Let’s reuse the example course in the following figure to discuss a courseware initial plan.

Figure 44. Courseware Example Scope

For this example, if you assume that figure depicts the courseware scope, then the courseware initial plan would look like the following.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Agile in software development calls the initial plan a release plan. We will call it a Courseware Initial Plan.|

Course name: XYZ Course

  1. {color:#000;}Iteration 1

–Courseware Increment #1 will include LOs 1–10.

  1. {color:#000;}Iteration 2

–Courseware Increment #2 will include LOs 11–20.

  1. {color:#000;}Iteration 3

–Courseware Increment #3 will include LOs 21–30.

  1. {color:#000;}Iteration 4

–Courseware Increment #4 will include LOs 31–40.

It doesn’t need to be any more complicated than this. We have used this level of detail with external customers.

You can then take this plan to the customer as the high level or initial plan. Be sure they know that iteration planning meetings may need to shift LO scope around to adjust to changing circumstances and unexpected events.

Iteration #1 will have a planning meeting that will result in an iteration goal that commits to a particular scope. If nothing changes in this example, that scope goal message to the customer will simply state, “LOs 1–10 will be completed during iteration 1.”

[]10.8. Don’t Start with Velocity Tracking

Many of the software people that use Agile recommend that teams track the velocity of the team to know how big of a scope they can realistically take on each iteration. Others warned of how management tries to use the velocity metric to compare teams, which it is not designed to do.

As it turned out for us, we had to use earned value management reporting anyway and had to meet the estimates made up front to be profitable. So for these situations, we did not need a velocity metric, so we ignored it. We had to impose the pacing on the teams. We had to explain why we were not hitting the goals any time we missed one. This is not ideal, but it was our situation.

This is why from our perspective, the cadence of the two-week iteration and the iteration goal of X number of LOs per day was what helped the entire team reduce the costs of tracking where everything was. The structure of the visual board helped minimize the transaction waste of the daily standup meeting. The shared accountability helped the team manage itself to a higher degree than we’ve been able to accomplish on other projects.

It just turns out that a velocity metric did not really help us. So we ignored it. You will have to determine that for you.

One of the really strong ideas behind Lean-Agile is that if something is not working, change it. It is not a dogmatic approach of specific practices. If velocity works for your team or sets of teams, then by all means use it. For our circumstances it did not provide value, so we dropped it.

[]10.9. Specialization Expectations for Training Development

Specialization is already common in training development teams in which we have participated or led. The instructional/LX designer often leads the work, often supported by other instructional/LX designers, a media person or a media team, and one or more developers that encode the instructional content, build software driven components, or modify the courseware “player.”

As individuals, we are often hired for our specializations. However, we need to be open in Lean-Agile to work outside of those specialties to help the team achieve better throughput. For example, during QA content checking and QA functionality testing, when putting the components or courseware through the QA checklists and test cases and identifying defects, all team members can perform this work. Lean-Agile is not about zero specializations, but rather it is about people being able to do more than one task well in the product development process.

Rather than trying to generalize too much, however, it is often faster to let the specialist do the thing they excel at because they are faster at it and it may produce less rework. For example, instructional/LX design is something that we have found not all people do well from scratch. Some have been involved in training for a long time, especially as instructors in workplace training. However, their instructor background was often primarily course conducts rather than developing new courseware from scratch. Modifying someone else’s fully developed course is easier than developing a course from nothing, synthesizing content from SME interviews that supports well-crafted learning objectives. So we have learned to ask during interviews about their experiences and challenges starting from zero to better sort those who can newly create from those that are better at revising.

Another challenge of generalization is the pay rate differences. instructional/LX design often pays more than media development. And software development may pay even more than either depending on experience and the technology involved. All of these roles use design thinking to varying degrees with different focal points, instructional work products, media work products and software products. We have had success with people who have multiple specialties, especially with game engine development for training courseware. So instead of thinking pure theory, adapt Lean-Agile to fit your organization’s situation and circumstances. If there are significant pay rate differences, you may need to adjust who is on the project team and move the highest paid specialist to another project without as much generalization. Your decision points may vary depending on how tight of a financial ship your organization runs. Some groups have much less a sense of how much they are spending than others on a monthly or sometimes weekly basis. If your group has to use earned value management and check costs weekly against plan, or if your group developed a plan that moved specialists away at a particular point in the project, then you will have to adapt Lean-Agile to your circumstances. Make a testable prediction, or hypothesis, and conduct some experiments to see what the data tells you is the best for your situation.

[]10.10. Why We Call it a Courseware Increment Instead of a Lesson

Historically, training development has addressed chunks of a course as the course, or as lessons. Some might wonder why we can’t just use these familiar handles to talk about the courseware.

We have used the following figure to show customers how the entire course is too large of a batch size. It graphically shows that even lessons tend to vary in length and do not support the Lean idea of consistent flow through our development system. LOs vary in size too, but they are more often similarly sized than lessons, so we have decided to use LOs as the work item level in our product breakdown structures. It is good enough and both the training teams and the customers grasp it quickly.

Agile, in our Scrumban style, fixes the schedule to two-week increments so the time does not vary, but the scope may vary. In this example, we have fit a certain number of LOs into a single iteration, so that becomes the batch size. So what do you call that? Agile calls it a product increment. So we’ll call it a courseware increment. That gives us a handle with which we can address this chunk of the course in our communications with each other and with the customer stakeholders.

Figure 45. Agile Increments are Similarly Sized vs. Lessons

The things delivered each iteration include:

  1. {color:#000;}Iteration Goal (scope commitment)

  1. {color:#000;}Each LO storyboarded and developed
  1. {color:#000;}Increment Test Bank
  1. {color:#000;}Fix Change Requests
  1. {color:#000;}Iteration Review/Demo (inspection & evaluation) in a synchronous meeting

[]10.11. Lean-Agile Works for Related Product Development

Lean-Agile works for related product development too. This can include higher end simulations and games that use 3D models and game engines. For example, moving to daily builds of game engine code instead of the prior practice of much less frequent builds. Other related product development can include complex simulations that need software code crafted for the training product. It works for eLearning, mobile learning and blended instructor-led training learning experiences.

The Lean-Agile approach can even help improve technical writing projects. The smaller batch size for tech writing would change from the manual or chapter to the data module (for S1000D), or the topic (for DITA), or the work package (for MIL STD 40051), or some similarly-sized chunk for custom schemas.

[]10.12. Degree of Formality – CDD or a simpler List of LOs and Strategies

In some organizations, they simply list the LOs and get to making them. In other organizations, especially those who have to deliver courseware to customers, the customer wants a step for seeing that the design will meet their needs.

Because training has been historically driven by learning objectives and their associated Bloom’s levels, for appropriate content and test items that align to the LOs, many organizations have come to expect or require a Courseware Design Document or CDD.

Some customer organizations spend small amounts of time reviewing LO scope and sequencing with particular emphasis on the evaluation or assessment strategies planned. Then they expect a collaborative session with their stakeholders on the plan to implement the LOs with particular instructional strategies.

Some customers just want the CDD at the end of the instructional/LX designer’s work.

Some organizations spend lots of time on these documents up-front.

The ideal Lean-Agile approach emphasizes prototypes over design documents, but you may still have to make a CDD for some customers if unsuccessful at convincing them to drop CDDs. First attempt to persuade the customer stakeholders that prototypes produce less expectation mismatches than design documents. The following figure shows how textual CDDs can be misunderstood and agreed upon even when the text may be interpreted to a different mental image by different readers.

Some buyer stakeholder’s actions seem to indicate that they think a “perfect” CDD will reduce the risk of the project coming unraveled. We have seen some organizations focus excessively on the design document out of habit and end up spending valuable resources creating many versions of the CDD. This is what Lean calls waste, but you may not be able to get everyone to agree to that. While fiddling with documents for days or weeks, there is no courseware prototyped yet, so the CDD only has the lower fidelity that text and images provide. Text can be, and often is, interpreted differently by different parties involved as shown in the following figure.

Figure 46. Everyone Agreed the Design Text was Great

This situation is analogous to Realtors telling you to use fresh paint because some buyers cannot envision a beautiful home when your paint has hand prints and smudges all over. Some stakeholders visualize differently when decoding text to their own mental image. Working prototypes provide a higher fidelity to the intended courseware design. The following figure depicts the prototype better representing what was meant and how excessive CDD effort may not reduce risk as much as we have thought in the past, especially when newer courseware has expected behaviors that are harder to depict with text.

Figure 47. Prototypes Demonstrate with Less Misinterpretation

Design is important in courseware products. We do not mean to malign the CDD. We simply emphasize that Lean-Agile emphasizes prototypes over documentation. How can you do that? What is appropriate for your organization’s needs?

If you make a CDD, pick a maximum time to be spent on it and stick to it. Then move to prototypes. Do more than one prototype if needed to get aligned between the customer’s problem and the proposed instructional solution.

Other alternatives may include a more bare-bones scope and sequencing CDD, one that lists the LOs and their sequence with the assessment strategy instead of a 100+ page CDD that elaborates on many things in text.

Remember, you’re going to build the courseware in small increments and have time to correct misunderstandings as you see how the instructional strategies are playing out in each iteration, so you don’t lose the ability to get what you want by pushing more of the design from up-front to distributed to the iterations, a little at a time.

If you have flexibility, try experiments on what is minimally acceptable, and if it works, go with it. If you have detailed contracts with requirements lists, try to influence the customer before the contract stage so the contract does not force both parties into spending more on the CDD than may be needed.

[]10.13. Translating Training Agile Terms for Software Developers on your Team

If you have software people on your development team, they already may be familiar with Agile terms. Then you may need to explain to them that for courseware, we have had to modify Agile somewhat. Because we apply both Agile and Lean, other terms will be there that they may not have used before.

For example when using Scrum, instead of sprints, for training practitioners we just say iterations. Instead of a Scrum Master, we just say Lean-Agile Coach. It has worked for us.

Emphasize that the reason you tend toward the more generalized terms is that you’re using a hybrid approach of Scrum and Kanban and Lean. So we’re not sending people off to a paid Scrum course to be certified. The differences in terms are minor, so the translation should not impede their ability to catch on and contribute quickly.

[]10.14. Content Versus Players

As technology continues to be included in courseware, people involved in learning experience development may benefit from conceptualizing and talking about learning experience content separately from players.

For example, for older Flash animations, learners needed a SWF player to see the animation content. For HTML5 animations, the learner needs a browser “player” that is HTML5 compatible so it doesn’t rollback to lesser content. Video content requires a video player. Audio content requires an audio player.

So now extend that concept to include the courseware player. SCORM content needs a SCORM player, a full learning management system (LMS) or an app that works like a limited LMS to play the SCORM package per the SCORM manifest.

Why this conceptual pattern helps is that we can think differently about the content versus how it is rendered for the learner in various players.

For example whether you use a commercial authoring tool or an open source tool like Adapt Learning, your eLearning content may be played on a desktop computer with a large screen size and a browser, or a mobile device with a small screen size and a browser. The rendering of the components on a “page” will vary depending on the device screen size. You may already be using responsive design to design for mobile first and large screens second.

That the “page” is no longer the atomic unit of courseware, especially with mobile learning, impacts how we think about and design training. For two dimensional courseware, “pages” are now constructed by combining a wide range of interactive components in any number and mix required. Players now use a scrolling page layout that adjusts to the screen size. Even HTML5 now has articles as children of pages. Add to that components, and you can adjust the layout as needed. The fixed-position “page” drops from our collective prominent mental model to now mostly on PowerPoint slides and print CSS versions of courseware.

The Adapt Learning courseware breakdown structure is a useful model for thinking of chunks of the courseware. It includes:

  1. {color:#000;}“Pages” (what a page is has changed with varying device screen sizes)

  1. {color:#000;}Articles
  1. {color:#000;}Blocks
  1. {color:#000;}Components

Another example is 3D immersive environments. The content includes 3D models of things and people as well as messages for the learner to explain their mission and provide tips and feedback. The player for the courseware is typically a game engine viewer. The custom-designed user interface (UI) for your learner will benefit from a UI tutorial so they can gain competence in operating your courseware.

A blended component of a desktop simulation, for example, may need a particular type of software installed to run for the instructor. Content needs a player.

Training teams can benefit from encoding courseware content in a format that can be automatically converted into various output types depended on what is needed, using XML or JSON for example. Today many training groups still use proprietary software to encode the courseware content. If the courseware has a long life cycle, can the encoding from LCMS-1 authoring tool be exported and easily updated for LCMS-2 authoring tool? Does this matter to your organization yet? Will it in the next three to five years?

Agile can be used by both the content development teams and the player development teams.

[]10.15. Communicating Kanban Defect Cards to the Customer

If the customer does not have their own way of tracking defects to resolution, invite them to your virtual Kanban board and let them see the way your team tracks it.

Less often, the customer has their own defect reporting system and that they require you to use. In these circumstances, it can help to create a Kanban defect card for each of the customer’s change requests or defects reported so your team can still visually track the defect resolution to closure.

Even if you have used your own defect tracking software before, you may still want to put a Kanban card for each defect on the visual board.

This puts all the data the team needs on the Kanban board for their daily standup resource allocation decisions about who will pull which cards to work them. If the team sees a large queue of defect cards, they can start fixing them and moving the cards to the customer validation queue.

Even if you retain your original defect tracker app, the visual card helps keep the workload visible. Our experience is that the customer notices when we fix an issue from the prior iteration, and they don’t have to continue to worry if we’ll fix that for them. It builds the customer’s trust in your development team.

Some training development team members initially complain about having to create so many Kanban defect cards/sticky notes. It provides a good opportunity to discuss why so many defects were reported by the customer at the last iteration demo and how can we do better going forward. It also provides a teaching moment to coach the team on why we track defect demand, a lean waste, separately from value demand.

Again, alternatively consider using 25%, 50% and 75% columns for defect progress with a single defect card per iteration traveling through them. As long as your solution makes defect demand visible, use what works best in your context.

Jeff Sutherland talks about one car company being compared to Toyota. He said the other car company made so much defect rework that Toyota could build another car in the time it took the other car company to fix the rework they made. His example points out the competitive impact of Lean-Agile on your team. Even if you don’t compete for business, you compete for annual budget, perhaps as a cost center. Be able to help your boss tell a good story to the executive team.

[]10.16. Simulation Development Uses Agile Like Software Development

As technology is injected more and more into courseware, there are additional benefits of Lean-Agile.

Getting instructional/LX designers and media staff used to Lean-Agile moves your teams farther when more interactive courseware is desired and simulations are added as either components of the courseware or as the primary delivery.

Simulations are far more dependent on software than traditional instructor-led training, so most of the typical software development ideas for Agile are not changed much for Sim development for training.

Historically, there was much focus on Adobe Flash and ActionScript as the language for development. Now because many mobile devices do not work with Flash, the focus has been towards HTML5 (JavaScript, CSS) at least on the front-end.

In days past, when the software components were small additions to a largely text-based course, the instructional development team would make a request for a segment of interactivity and they would treat it as a black box. The request would go in to another team. The instructional development team would get little data on the progress until the development was almost done, other than some questions from the interactivity specialists.

As we move into instructional/LX designs where the software driven components make up larger and larger portions of the total courseware, it is no longer viable to treat them as black boxes and hope they turn out on time and under budget.

In plan-driven, traditional training development, the uncertainty of the predicted end date of a simulation stays high until nearly the end of the project. That is because the courseware or simulation “batch” is not finished until near the end, increasing risk. Some organizations use lesson “batches” which are smaller but their size can vary significantly. Little data is available on how long it takes to complete these larger batches with this particular team of people and their tools until late in the process. When using Lean-Agile, the data comes in earlier. A product increment (of courseware or of simulation) is completed every two weeks. For each iteration, the uncertainty of the end date decreases.

Visual management also helps make the software-driven components that are still in process visible to the learning experience development team. This visibility also helps professionally develop instructional/LX designers by increasing their awareness of all that is involved in building software-driven learning components for courseware and helps them to better judge what is feasible in a particular timeframe and budget when it comes to instructional components that are largely software. Implementation has changed too. The courseware product backlog might benefit from a simulation component backlog so that the work items for the training development team are not all mixed together with the work items for the simulation development team.

For example, the work items are user stories, written in the user story format:

“As a [role], I want [Sim behavior] so that [benefit].”

The interfaces for both the learner/player and the instructor (if facilitated) are treated the same as in software application development.

Developing an underlying systems dynamics model or discrete event simulation model to drive learning is a large batch of work when considered as a whole, and it needs to be broken down into work items that can be completed during a single iteration.

Learning objectives still constrain simulation experience goals so LO work items are still used.

Work items to develop learning content that is accessible by the simulation player in a specific context during the Sim is another example of the overlap between the Sim development team of software developers and the instructional development team.

When the customer wants the ability to maintain and modify the simulation content, the encoding of most of the content in HTML5 with simulation data in separate XML or JSON files separates it from the simulation engine code and allows non-specialists to modify what shows up on the screen. This allows customers to keep the Sim relevant for a longer period of time.

So Lean-Agile also benefits learning simulations and other software-driven components by making the process and the current progress visible to all stakeholders, including exposing instructional/LX designers to more complex technology development practices.

[]11. Adaptations for Customers that Reject Agile[]

[]11.1. Using Kanban In-House Even if a Customer Requires Waterfall

In one situation where we needed to have the staff for a customer not using Lean-Agile to be conversant with it so when they moved from that project and back to a Lean-Agile customer they could help, we decided to have even the traditional ADDIE waterfall team use a Kanban board in addition to their normal process.

This customer was unaware we were using it in-house because we did not show it to them.

We set up the columns differently than the Lean-Agile board because for this project the storyboard was for an entire lesson at a time and the entire course had to be storyboarded before we could move into any development.

We also did not have Scrum fixed iterations on this team, but used the Kanban board to see the progress towards the traditional courseware schedule milestones on the Gantt chart.

Guess what we learned?

The Kanban board and the daily stand-up synchronization meetings of 15 minutes maximum still helped keep the team better coordinated.

It did not help as much during the CDD development because the instructional/LX designers felt they already knew what they needed to do and the Kanban was additional overhead work to them. However, it helped us see where multiple teams were with the work in process at a glance, so we asked them to do it for our benefit rather than for theirs. They did, albeit some in a grumpy way initially.

When we got to storyboarding and the authoring tool and experts had to come aboard the team to get the ID’s vision into the tool, it helped more.

After that, when we went into that customer’s “developed lessons” stage, it helped just as much as on the Agile teams because everyone involved could see what was going on.

Overall, we recommend using Kanban for just about any team doing work. The only time it may not be worth it is when the team only requires one person for a time.

[]12. Troubleshooting Agile Problems[]

[]12.1. Diagnosis Questions

When diagnosing problems with Lean-Agile consider the following questions:

___ Does the customer and contract expect and allow for scope adjustments?

  1. {color:#000;}Prioritize the product backlog items with the highest priority items on top. This helps ensure that each iteration pulls an iteration’s worth of product backlog items that are of the highest value to the customer. This is a key part of the risk mitigation of Lean-Agile.

  1. {color:#000;}If the customer or contract requires fixed-scope, then all product backlog items may end up being the same priority.

___ Has the team’s overall cycle time increased?

  1. {color:#000;}Reduce the number of things in process (reduce WIP inventory).

  1. {color:#000;}Improve the average completion rate.
  1. {color:#000;}Reduce rework. Make a visual signal for rework like a different colored card. We use orange virtual cards or sticky notes for CRs and defects.
  1. {color:#000;}Make blockers visible (we use red signals) and reduce blockers actively, escalating as necessary.
  1. {color:#000;}Are any work items too big, that should be split into smaller work items?
  1. {color:#000;}Throughput allows forecasting future capability. Throughput is the rate of delivery of customer-valued work into production.

___ Is the WIP inventory increasing?

  1. {color:#000;}Are WIP limits exceeded? Are WIP limits configured correctly for this team?

  1. {color:#000;}A backed-up WIP queue is not a matter of opinion. Look at the board for evidence.
  1. {color:#000;}This is a leading indicator. An increase in WIP today means an increase in the time to deliver that work in the future. You cannot do more work than you have the capacity to do, without taking longer to do it.
  1. {color:#000;}Can you free up resources to address the WIP inventory? Swarm if needed.

___ Is there anything that delays the start of work, interrupts work, requires ramp up to “full speed” or is similar to another task?

  1. {color:#000;}See if any of the interruptive or delaying tasks can be off loaded. The goal is to have people speed through value added work without delays or interruptions.

  1. {color:#000;}What ideas does the team have for reducing latency?
  1. {color:#000;}Can any of the interruptive or delaying tasks that can’t be off loaded be automated or streamlined?

___ Can complexity be reduced?

  1. {color:#000;}Standardize internal tasks like Lego blocks to reduce task complexity.

___ How can we further reduce transaction costs?

  1. {color:#000;}Smaller batches mean more transactions, so lean out as many transaction costs as possible.

  1. {color:#000;}Reduce coordination costs with scheduled fixed time box meetings for synchronization daily.
  1. {color:#000;}Reduce coordination costs with long meetings with everyone present.

[]12.2. Experimenting with PDCA

We all hope customers know exactly what they want when starting a project, but sometimes their needs change as what we do helps them achieve more clarity. Our older approaches tried to restrict the customer from changing while Lean-Agile approaches can allow changes even on fixed-price work to the degree the customer is willing to swap out an equal amount of now-lower-priority work in its place.

In working with large, complex projects that inevitably change, we have found that the Deming Cycle works well because it works on an empirical basis, meaning practitioners verify by observation or experience rather than by theory.

The Deming Cycle steps include:

  1. {color:#000;}Plan

  1. {color:#000;}Do
  1. {color:#000;}Check
  1. {color:#000;}Adjust

Agile methods are empirical, or evidence-based, using check and adapt cycles, or what Scrum calls inspect and adapt cycles, to adjust team actions and plans based on principles to the emerging situation in your specific context. Every iteration has a reality check, providing more reliable predictions of milestone deliveries.

After using PDCA to make an improvement, standardize the new innovation that changed your existing practices so that you retain the gains. Then use PDCA again for the next gain to continually improve.

Figure 48. Retain the Gains by Standardizing as you Improve

Let data inform your decision making. There are uncertainties when trying something new. Applying Agile the first time is new, and all eyes may be on you and your team.

Set expectations so people expect to use experimentation at the beginning. Use the Deming Cycle. The PDCA cycle allows you to go forward without knowing everything up front. Socialize your plan. Apply the Lean-Agile approach. Check your data. Then adjust to overcome challenges and to capitalize on opportunities.

The ‘P’ can also mean predict. Experiments are important during innovation to see what happens. Our testable prediction, or hypothesis, is that Lean-Agile will improve cycle time for learning experience development.

Focus on making improvements in how well you implement the principles. For example, does your focus on flow, rather than focusing solely on waste elimination, prove a better catalyst for continuous improvement for learning experience development?

[]12.3. Until Team Members Get Pull – Overcoming Push

People new to Lean-Agile don’t initially get the pull idea when you first tell them about it. Conceptually they follow you, but it tends not to be reflected in their performance right away.

It is not that we are manufacturing lines, but it seems that most people want to do “their” work and pass it along at first. It takes time to get that we mean it about optimizing performance for the entire team, not just for the individuals; and even though we hire specialists, we want them to generalize where possible to help the team move cards along using one-piece flow, doing one thing at a time.

We have found it takes a week or two of noticing cards in the wrong columns, or noticing that they are overproducing at their preferred station and pushing their cards to a queue for someone else.

After a few weeks of coaching teams, they all adjust pretty well to the idea of pulling cards. It helps that we walk the board from right to left, emphasizing the one-piece flow.

Once the team gets it, we can move on to other areas to coach.

Having said all of that, there are a couple of exceptions where having a function to move a bunch of cards all at once is a nice feature on a virtual Kanban board app. When we queue cards as “ready for customer demo”, after the demo, the entire bunch moves to the next column. This function for moving many cards at once is handy to reduce transaction cost waste.

[]12.4. Fixing Bottlenecks

In Los Angeles, California, the traffic can get bad. One day a while back someone heard about the latest traffic jam from the traffic helicopter seeing it all from above (like on a Kanban board), and some Engineer decided to apply a highway on-ramp stop light to meter the flow of new cars into the highway. It helped reduce traffic jams.

This throttling of incoming cars also helps with work items when developing courseware. The metering device is not a stop light at the on-ramp, but a similar acting idea called a WIP limit. This limits a particular column (process station/step) to a certain number of work items (rather than cars) that can enter the Kanban board (rather than the highway) at a time.

Florida has evacuation routes for hurricanes. The road signs indicate that during evacuation, all lanes are used for out-bound traffic. This lessens the impact of traffic jams that result from everyone fleeing the hurricane at the same time and so many cars getting bottlenecked on a two lane road. This idea of adding additional resources, sometimes called swarming, to help reduce traffic jams, can also help in your learning experience development bottlenecks.

These are simple ideas. If you want more detail on how to fix bottlenecks, read The Goal, by Eliyahu Goldratt and Jeff Cox and then study Goldratt’s Theory of Constraints. We apply the Theory of Constraints, and it has workable approaches for resolving bottlenecks. It applies to courseware production as well as manufacturing, the example that Goldratt uses in his book. When reading his book, you will have to interpret the production principles from manufacturing to the courseware domain.

[]12.5. Dealing with Customer Demand Fluctuations—Determining Labor Capacity

Buyer/Customer demand is not always even flow. Sometimes they ask for a lot at once and other times they slowly decide while we have people on the bench waiting for a go signal. Let’s discuss a few heuristics.

One way of dealing with that is only using 85% of your capacity to leave a 15% reserve. This reserve can act as a buffer for the shocks to your system that come from a sudden surge of work demand.

You have to request management support for this to be a survivable approach because the cost for this "reserve" will be noticed and because the historical focus on maximizing staff utilization may prompt leaders to ask you to increase staff utilization from say 85% to 95%, or worse, 100%, leaving little to no reserve capacity. Reserves help even out demand spikes.

The other possible solution is to hire temporary staff as needed for the brief surges and keep your core staff from being overburdened. Also, try to reduce the delay time for hiring to as low as possible. If normal hiring cycle time is 50 days, try to reduce it. A hiring delay of only 15 days has less impact on your labor capacity than 50 days hiring cycle time. You may need a good relationship with the recruiter you use too.

[]12.6. Accounting for Variances from Plans

Some buyer stakeholders are more open to pure Kanban and flow. Some buyer stakeholders prefer to pretend that the plan, that included significant guessing, is now written in stone. These types of organizations focus a lot on accounting for variances from the plan.

Saying you did not know enough about the situation early on when the plan was made is not generally considered an acceptable rationale for a variance in these organizations.

So, you may have to account for how many hours and monetary units (dollars, euros, etc.) that the team is off from what was expected and explain why that is.

If the person with the pay rate you had intended was not available and you had to use a higher paid person instead, your monetary unit variance will be off due to the usage of this higher paid person.

If your instructional/LX designer gets jury duty, you may end up explaining why you had a variance against the schedule.

Some stakeholders spend much time and energy doing this type of comparison on a monthly basis. Some do it quarterly. Some do it annually. Where possible, lean this process.

Lean-Agile will not help you with plan-driven processes and control mechanisms. Assess how much this will impact the team and who will have the role of collecting and reporting this information.

[]12.7. Some Customers are Not Used to Fixed Time Box Iterations

Sometimes a customer would ask us, “Because our SMEs are not ready, can we move the iteration review date out a few days or a week?”

Then we spent some time reminding them that one of the most effective parts of the Lean-Agile approach is that the development team has a consistent cadence or pace to work. We explained the impact of delaying the courseware deliverable date and reminded the customer that if we need to shift some of the specific work items from iteration #2 to iteration #3 we can do that to accommodate the SME availability without breaking the fixed time box iteration mechanism that provides the beneficial cadence to the development team. The customer was okay with us moving a few work items those SMEs needed to be available for to another iteration and bringing forward some of the work items from the next iteration instead.

We held the iteration demo with all the working-ready-for-demo work items, did not demo those work items that were unfinished and moved to another iteration.

[]12.8. Variance is Not the Enemy in Product Development

In Lean manufacturing and especially in Six Sigma efforts, there can be a focus on finding and eliminating variance. When manufacturing widgets, the idea that all widgets should be the same is appropriate. For courseware, this is a bad outcome.

When you are creating one-off courseware, eliminating variation should not be the focus. If you are making multiple courses for a single customer, you don’t really want every course to be almost exactly the same. Sameness in user interface is good for minimizing learner confusion. Sameness in content and instructional strategies introduces boredom quickly, reducing the effectiveness of the courseware and negatively impacting the intended learning outcomes.

We need to be able to plan, so we really want to know what does it typically take our team to create low-complexity courseware, medium complexity courseware, and high complexity courseware.

We can do this by tracking productivity metrics over time in each area and using those historical figures to predict the future to a degree, or at least reduce uncertainty somewhat.

Just don’t forget the maxim in warfare, “No plan survives first contact with the enemy.” Planning helps prepare but rarely predicts how the entire endeavor will play out.

[]12.9. Using Queuing Theory to Improve Kanban Traffic Jams

Sometimes when you run out for lunch, one restaurant has 5 cashiers and 1 cook.[64] So you get your order in fast, but then you wait almost your entire lunch time to get your food. Another restaurant has 5 cooks and 1 cashier. Here you wait a long time to make your order, but then your food arrives very fast. This last example is a single queue system. It is stable before the lunch rush when the cashier’s capacity can handle one or two customers. When 40 people arrive on a bus, in a large batch, the restaurant has a bottleneck, a traffic jam. Some customers leave rather than wait in a long queue because:

  1. {color:#000;}The queue is visible to all involved.

  1. {color:#000;}Waiting in a queue is not valuable to customers.

A queue is stable when it does not grow to become infinite over time. The queue system is stable if the mean service time < mean inter-arrival time. On any road, the queuing becomes unstable after a car crash. Then the queue of cars lengthens and lengthens.

One store we went to had long single queue, with multiple cashiers, or servers. That went pretty fast. Wal-Mart stores tend to have multiple queues each with a single server.

These are examples of queuing. On the highway we call it a traffic jam. In the restaurant we call it a frustratingly long line.

Queuing theory helps minimize customer time wasted or work item time wasted in queues or waiting lines, pending a server or a service.

Little’s Law [65]

Cycle time is work in process (WIP) divided by throughput.

Cycle Time = WIP / Throughput

Cycle time is an important metric in queuing theory. It consists of the sum of the queue time (spent waiting) and the service time (spent adding value by working on it).

For a given arrival rate, the time in the system is proportional to customer occupancy.

Little’s Law tells us that the average number of customers [in line] L, is the effective arrival rate λ, times the average time that a customer spends in the [line] T, or simply:[65]

N = λ T [65]

The ready queue contains a queue of work item cards ready to be started. It is a queue because the work item cards are “standing in line,” with the first card at the top being the first serviced. When started, that card leaves the ready queue, and all those remaining in the queue will shift up.

Bottlenecks happen at access points and result from overloads caused by high load. For learning experience development, the bottleneck of our system is the workstation or process step with the lowest throughput. The longest queue shows you your bottlenecks. Read Goldratt’s Theory of Constraints for more detail on this. Since the other workstations or process steps complete their work at an higher rate by definition, they push their completed items to that station which is unable to process it with the same speed. The bottleneck is the most important place to focus improvement attention because it dictates an upper limit for the throughput of the entire system.

Little’s law tells us that measuring queue length is equivalent to measuring cycle time as they are proportional in a stable system.

Variability in rate of arrival or in service time increases the cycle time and the queue. Larger batches take longer to service, so moving to smaller, more similarly sized batches results in decreased cycle time. Large batches make queues grow, and by Little’s Law, increase cycle time.

To improve cycle times and reduce queues, target an even, stable rate of arrival, like restaurants do by introducing discounts at certain times of the day. The LA traffic lights restrict how many cars can enter the highway, making the rate of arrival more stable.

Queuing service choices include:

  1. {color:#000;}FIFO: First In First Out (generally not used in Agile)

  1. {color:#000;}PQ: Priority Queuing (used in Agile)
  1. {color:#000;}Lean pull-based queue systems (used in Scrumban as a sub-method to priority queuing)

An example of priority queuing is airline check-in. Typically there is a first class queue and a standard queue. If you have a first class ticket, you have preemptive queuing and do not have to wait. So it is with queuing for Lean-Agile. You can prioritize some work items as higher priority than other work items.

The throughput of a training courseware production system is the average number of finished products per unit time. Products here mean work items on the Kanban board, typically learning objectives.

The rate of arrival is the amount of work entering the system in the unit of time–for example, the number of work items you accept for an iteration. To minimize cycle time, using Little’s Law, the rate of arrival should even out as much as possible. This is why we drop the batch size to Learning Objectives instead of Lessons.

How you choose to handle queues can have a large effect on the team’s performance. You may need to have service differentiation. Many Agile teams in software development use priority queuing, so the work item card at the top of the queue gets pulled or served first.

Queuing theory explains why WIP limits on our Kanban board help to reduce waiting waste and work item bottlenecks.

Handling and tracking partially finished larger batches costs quite a lot more than managing smaller batches of work items which are not started (in a queue) or are completely done. Leaning out these costs speeds development cycle time.

Reducing the batch size from one very large batch (course) to five or six iteration batches (increments of LOs) has large positive effects. This why even early efforts at applying incremental methods usually cause significant improvement, despite not having the rest of Agile or Lean working yet.

[]12.9.1. Queue Summary

  1. {color:#000;}A queue or buffer between work stations (process steps) absorbs timing variation.
  1. {color:#000;}The queue itself should have a limit, so that if the queue fills up, the upstream producers will halt.
  1. {color:#000;}The queue allows for slack. Optimum flow means just enough slack.

[]12.10. Capture Troubleshooting Lessons Learned for Other Teams

This topic may sound reasonable enough. Things got busy, and we forgot to do this once. When two teams both worked for a single customer, the customer got used to us performing a certain way with them from the first team they worked with. Then when the second team entered the scene, they did not know about these “ways” the customer had already worked out with the first team in their iteration demos. There were a few uncomfortable moments with the customer as they realized we had not cross pollinated these lessons learned from team one to team two.

We don’t recommend you go through that. To avoid it, share the lessons each team learns about the same customer.

While you’re at it, share the lessons each team learns with the whole Lean-Agile approach.

One of the instructional/LX designers suggested having a monthly idea sharing time with all the team instructional/LX designers so they could bounce ideas off each other, brainstorm and talk with other instructional/LX designers. This verbal lessons learned sharing was seen as valuable by most of the instructional/LX designers.

[]12.11. Troubleshooting in Obvious, Complicated, Complex and Chaotic Domains

Because in complex systems no plan survives contact with complexity, we mitigate the impact by first, seeking to understand principles and then tailoring our methods; and second, by identifying which specific context we find ourselves. Then we can tailor our approach for that context.

We know of a person in Austin, Texas who has a black belt in a martial art called Taekwondo. He reported that his Sensei had mastered multiple types of martial arts, had achieved mastery over each, earning the highest belts, and only then did he begin to modify the art and teach our friend. If we get the principles and theory behind the various Agile methods we observe others using, we can adjust appropriately for our context, which may be different. Just like mastering a martial art, this takes effort to see behind the method and look for the principles and theories. It takes study, reflection and practice rather than following the loudest or most repeated consultant. This is why Lean-Agile is a big skill.

The recent gains in the science of Complex Adaptive Systems, or complexity science, offer new understandings since the days of yore with Frederick Taylor’s scientific management theory as a companion tool to Lean-Agile. David Snowden’s Cynefin Framework [66] provides a helpful tool for choosing responses that are appropriate to the context for viewing a multi-dimensional and dynamic reality. Interestingly, David’s history of his framework [67] describes his early thinking about this framework as an ontological model for knowledge and learning. And we’re applying the framework to decision making in pursuit of improving learning experience development.

The following figure shows the Cynefin Framework diagram.

Figure 49. David Snowden’s Cynefin Framework [68]

The Cynefin Framework is another tool in our tool bag, that informs our Lean-Agile efforts, as does the Theory of Constraints.

Why does this apply to learning experience development team troubleshooting?

A [learning experience development] team is a complex adaptive system because it consists of parts (people) that for a system (team), and the system shows complex behavior while it keeps adapting to a changing environment. [69]

— Jurgen Appelo

Troubleshooting Lean-Agile problems in the Obvious domain of the Cynefin Framework typically means a list of recipes for solving common problems. The right action is obvious to the whole team, so just assign someone to do it. Examples include the 5 Whys. The cause and effect are linear and traceable.

Troubleshooting Lean-Agile problems in the Complicated Domain of the Cynefin Framework typically means advanced problem solving analysis. Assign the problem to an expert to analyze and propose action. Examples in this context include Kanban Boards, Six Sigma and Statistical Process Control.

Troubleshooting Lean-Agile problems in the Complex Domain typically means experiments and tests. Collect coherent hypotheses and ideas about what to do. Propose safe-to-fail experiments. Set boundaries. Use what Nicolas Taleb describes as optionality to amplify successes and dampen failures. The right answer is only clear in hindsight. Feedback from visual management changes the whole system.

The Chaotic domain is only a temporary transition state. Avoid chaos if possible. If you can’t, assign a leader and act immediately to restore order. Troubleshooting is delayed for a bit while leaders respond.

We see value in the Cynefin Framework as a way of better understanding where our systems are situated and orienting to make decisions about the most appropriate learning and development strategies and interventions. It provides a way of discussing what participants are see or intuit. The Cynefin Framework also helps in troubleshooting projects and even when facing a gap between actual performance and desired performance. This is an emerging framework, so further monitoring, application and learning is needed.

[]12.12. Other Troubleshooting

  1. {color:#000;}Problem: The team does not apply Lean-Agile well.

Solution: Start them using a Kanban board initially and delay WIP limits until they have matured a bit with the basics.

  1. {color:#000;}Problem: Organizational fixation with schedules makes it hard to use Lean-Agile Lean

Solution: Only schedule down to the iteration level and delivering an iteration goal (scope commitment). Combine storyboarding, developing, testing and fixing defects/change requests as a single “work package” in the schedule. This provides the instructional/LX designer the room to shift some learning objective work item cards between iterations up until the iteration planning meeting.

  1. {color:#000;}Problem: Team members are pulling 15–20 work item cards into their in process column at once.

Solution: Coach the team members during the stand up to only pull the number of cards they can actually work on. Explain how we will use the queue from the last process step to measure total queue time, which Lean calls waste. Let them know that they don’t get credit for any work that is partially done—called a 0/100 earned value technique if you are a cost account manager. Also help them get that the cycle time for all cards is faster if you use one-piece flow and take one down and finish it before you pull another Kanban card from the queue.

  1. {color:#000;}Problem: Team members use the list functionality of a virtual Kanban board application or write little lists on the sticky note on a physical white board.

Solution: Coach team members about how that choice optimizes their work for their personal level, but that we need to optimize for the entire team. Explain that we need to be able to see the card move between columns to see progress of the work. Our experience with the lists on a card is that you cannot glance at this team’s Kanban board and quickly see where the work is in process without opening the individual cards and reading the lists. This breaks our vision of visual management.

  1. {color:#000;}Problem: Team develops work item cards that are too big in size.

Solution: Show the team that this approach does not let us visually see progress for that work item in a couple of days, breaking our visual management system. Let them know that ideally we want similarly sized work item Kanban cards moving through the “system” or workflow so we can see a continuous flow of progress.

  1. {color:#000;}Problem: The team does not hit their iteration goal.

Solution: Explain up front that the team will start the iteration review/demo by covering what was in the iteration goal and reporting to the customer stakeholders if the team met this goal or not. If not, they must explain the rationale and their recovery plan. Being clear that this is the expectation tends to help teams focus their energies and deliver as expected. Then on the exceptions where events caused a miss in the iteration goal, the team explains the situation and how they will improve/recover.

  1. {color:#000;}Problem: The team presents a functional prototype that is not accepted by the customer.

Solution: Remind the customer it is better (less rework and cost) to get this feedback early than wait until much later in the development process to find this out. You’ll still have to fix the gap between what you delivered and their expectations, but it will not cost nearly as much to fix now than if we hadn’t gotten that feedback until later.

  1. {color:#000;}Problem: Customer-provided SMEs are not sufficiently responsive for the team to meet the iteration goal.

Solution: Escalate the issue to the SME leadership in the customer organization. Educate the customer during “kickoff” meetings about expectations so that the SMEs know up front that requests for information need to be responded to that day, not a week or three later when fixed time box two-week iterations are marking the cadence for the development team.

  1. {color:#000;}Problem: Existing employees with long experience in other approaches do not want to use the Lean-Agile approach.

Solution: First try to persuade them to support the change. Explain the organizational benefits and that we are optimizing for the organizational level, not the individual contributor level. Explain your vision for using this approach. Sometimes increased vision can lead to increased motivation. If they continue to be an obstacle, make the expectation clear that this is what is required of the employee. If necessary, take actions to correct the employee using your organization’s human resources processes, up to and including termination if necessary.

  1. {color:#000;}Problem: You are asked by the organization to show other disciplines how Lean-Agile Lean works.

Solution: This is a good problem to have. Use your ability with training development to create a brief version and a more detailed version of “how to” or offer to coach them in their first effort.

[]13. Mistakes and Misconceptions Others Made So You Don’t Have To[]

[]13.1. Team Members Working on Too Many Things at Once

There is an illustrative Dilbert cartoon that shows the pointy haired boss asking Dilbert to do project X. Dilbert replies that he already gave him Project Y, so which one is the priority. The boss says, “Can’t you just combine them and do them both?”

As companies get more efficient, they have relied on some heroes to do much. This is one approach.

The challenge to this is overloading. Some heroes do not say no, so their work in process inventory gets bigger and bigger. Little’s Law tells you that the hero’s throughput slows as more items are added. Experience validates this.

Little’s Law shows that fewer work items in process at once is faster when you measure the flow time through the process. Then those things are done, and you can get the next priority item.

Some people still believe in multitasking. They take on 4, 5, or 6 work items at once thinking they can switch back and forth between them and “multi-task”. This is bunk. But don’t tell them. Show them. Use an exercise to show this.

What really happens if you measure them is that they context switch. This context switching time adds up and their overall throughput slows.

The solution is WIP limits. This is hard to do in some organizational cultures. People, especially heroes, get pulled by many organizations to help “just for a little while” on this or that and the “little while” extends beyond estimates.

WIP limits for a single person should be set at two (2). This means that if work item #1 is blocked, then the person can work on work item #2 instead. But when you add more than that, their throughput will drop.

We have found that when first teaching Lean-Agile, this is a lesson that can come on another teaching or coaching iteration. You can get positive changes out of Lean-Agile without WIP limits initially. Then later, as the participants grow in their understanding of Lean-Agile principles, teach WIP limits.

Heroes don’t believe it at first. Your management may even initially think it is an approach created by slackers. But when you walk through the data, or better yet, get them to do a brief exercise, they start to trust the idea even if they think it sounds crazy at first.

When you conduct an exercise, consider that Jeff Sutherland has a good one in his book that we have used with organizational leadership teams. Another good exercise involves using a bunch of dice. These exercises are all over the Internet. Find the one you can pull off in your context and use it. It goes over better than a Power Point presentation.

[]13.2. Ensure your Contract Allows Scope Reduction by Prioritization

If you are in an organization that sells courseware, you will likely have contracts with your customers for the work.

One project’s contract required all the work items originally listed, and the customer wanted them in lesson order, so prioritization was predetermined. In that case, there was no grooming the backlog by prioritization. We just split up the stack of work items into iterations. Easy peasy.

If you have influence over the contract, we suggest that you bring up prioritization early in the customer meetings and get their buy in. Countless times the SMEs or stakeholders will change their mind on what they want as they start seeing it come together.

You can still offer fixed prices if your contract allows prioritization. Then the highest priority items make it in scope and the lower priority items may not.

[]13.3. When SME Delays Last Beyond One Iteration

In the case of two-week iterations, there will be some components of courseware that cannot be developed in a single iteration.

For example, it takes time to make a video. The person or object of the video capture session has to be scheduled and the video crew has to travel. Or consider the time involved in custom-programmed complex interactions with equipment that behaves like the actual equipment.

In these cases, we start in one iteration but do not get credit for the work item until an iteration or two later when we finish.

Lean-Agile meant that we did not have to plan out every detail of the video shoot months before the dates would be known. It saved overproduction waste on guessing when the person would have been available.

That flexibility allows the team to move more quickly through the planning phase and into designing and developing courseware.

[]13.4. Scaling to Many Teams Means Sharing Lessons Learned

When you have multiple or many teams all using Lean-Agile, it can be challenging to share lessons learned. We caught some of the same issues occurring in different teams, and it became clear that we needed to figure out how to share improvements more broadly.

Of course, at the team level, the team holds their Retrospective meeting, and they gain within the team. However, sometimes the people involved in a project are moving so fast they don’t feel like they have time to review written lessons learned in a repository. Lessons learned repositories are great for the organization’s institutional memory, but getting people to go there to search or even skim turns out to not be an easy challenge to overcome.

So one instructional/LX designer suggested gathering all the instructional/LX designers from all the teams once a month for an unstructured sharing conference call. We were then able to share between instructional/LX designers in different locations and help raise the awareness of key lessons learned with all involved.

The side benefit of a meeting like this is that as relationships begin to form the instructional/LX designers can reach out to validate an idea they have with others that have a similar specialty to their own.

Smaller organizations may not have the problem of scaling continual improvement. Smaller sized organizations are good at sharing as an attribute of smaller teams of people. Colocated teams can do even better because of the social interactions that occur in groups that work in the same location.

[]13.5. Good People are Always in Demand

Growth is a good problem to have. It sure beats decline and reductions in force or whatever current euphemism your organization uses for letting people go.

Growth also brings challenges. If you are growing when others are shrinking you can gain from their people. The ability for job postings to get broad exposure is much higher today than it used to be many years ago. High skilled people can be hard to find in some market conditions.

If you are a shop of experts who custom-craft every solution, you count on highly skilled people in a crucial way. Turnover can be difficult.

If you are a shop that has a core of super people and hires others not quite as experienced yet, you can grow the newer talent into super performers too.

We have found that we prefer people who have exposure to the key skills of building courseware and, more importantly, have some experience with creating and synthesizing content from scratch using SME interviews, and from low amounts of authoritative source materials. Where we tend to ask more questions is if someone has only demonstrated that they have facilitated someone else’s courseware. Instructor-led experience is great. However, some people come from that background having never had to create their own courseware. These folks can struggle and burn many more hours and not provide a good solution for your customer. Be careful with these.

Behavioral interviewing has been better for us than some other techniques. Unlike investing in stocks on the stock market, people do tend to behave in the future similarly to how they have behaved in the past. We ask for examples from their mental database of memories that show this or that. Then we listen. People tend to reveal what they have experienced versus what they understand in theory but have not done much or at all. The more they talk and we listen, the easier it is to spot highly skilled staff.

[]13.6. Some SMEs Don’t Know What to Expect from Lean-Agile

Some subject matter experts (SMEs) have little exposure to training jargon. They may not know what Bloom’s taxonomy is or how we may use it, but they know what knowledge and skills are needed to succeed.

In a similar way, we have found that many SMEs are not sure what to expect from a Lean-Agile approach either. It seems that in the domain of software engineering, many more people are comfortable with Lean-Agile terms and concepts. We have found that training people and SMEs that have already worked to support a software product project have often already been exposed to Agile or already know how to use Agile.

So during kickoff meetings, it can be helpful to SMEs to lay out what will happen in non-jargon terms. We prefer generic Lean-Agile terms like iteration to Scrum terms like sprint. We have found that Scrum jargon gets lost with audiences even more so than training jargon.

For SMEs who have supported training projects previously, they have often approached supporting Lean-Agile projects the same way they’ve done all the other training projects. We recommend briefly walking through the differences.

What does not change:

  1. {color:#000;}Unchanged is the need for SME inputs because they know the job-relevant details that successful performers need to feel more engaged in the courseware

What does change:

  1. {color:#000;}SME response times to instructional/LX designer queries need to be faster during the iterations to help the development team get the courseware done on time: hours not days.

  1. {color:#000;}Escalation paths and triggers need to be clearly laid out, so they know the consequences if they delay to the point where they risk an iteration.
  1. {color:#000;}You may not get to see the same content over and over because, ideally, after we’re done with the iteration or after the change requests from that iteration are fixed, we plan never to consider that chunk of courseware again.
  1. {color:#000;}Complex case studies or interactive activities that SMEs need to help develop should be treated as work items that will likely cross iterations by both the SME and the instructional/LX designer.

[]13.7. Kanban May Not Help as Much for Teams of One

We have found that instructional/LX designers are typically pretty good at managing their own work. Making their work visible with a Kanban board can be done. They will do it if insisted upon. However, the benefits with a one-person “team” are to the management more than to the instructional/LX designer, unless they are looking for a new method.

For example, in analysis projects where we may not employ a multi-disciplinary team, but only a single instructional analyst or designer, the project may not benefit as much from visual management with only a single contributor.

We are not saying never try it with teams of one, but that some experienced instructional/LX designers are not as open to using it for one-person projects because it is different and individual work can be tracked in countless ways. So given the cost/benefit ratio, we don’t push one-person teams too hard given our particular context. You get to judge what to do for your specific context. Let us know how it goes for you.

However, when the team size goes to two or more, visual management adds significant value.

We’re convinced of the usefulness of Kanban for one person, and we use them for our own single-person jobs that we do. However, the coaching, arm twisting and complaining of some instructional/LX designers may make it not worth the grief because the benefits gained with only a single person who does not have to coordinate with anyone else are smaller.

For those that do want to use Kanban for themselves, we suggest using concepts from David Allen’s book called Getting Things Done combined with a personal Kanban board to keep it visual. For that, you may want to use Trello, a great application on the iPad, iPhone, and from a PC so you can see your virtual board from any device.

[]13.8. Stakeholder and Organizational Resistance to Lean-Agile

Some customer organizations are light on process and just want the results. These types typically don’t have objection to Agile, especially if you only start with Kanban initially.

Some organizations are process heavy and document-centric for planning and artifacts. These types of organizations may resist Lean-Agile initially because it threatens to change their well-practiced processes.

Convincing an organization to trust you using Lean-Agile is a selling role. You are trying to influence a decision. Part of it is being trustworthy. If the customer has no experience with Lean-Agile and you’re asking them to step out on a limb with you, they have to assess how much risk they think they are taking on to do so.

There are benefits for them. You can introduce them. Some people are attracted to benefits and will move towards something. Lean-Agile also reduces risks.

Some people prefer to avoid and move away from something rather than to go towards something. These folks are easier persuaded by pointing out how they can avoid pain points.

  1. {color:#000;}We have described or shown buyer stakeholders a picture of the first time the two railroads met in the western United States, and asking what would have happened if the two tracks did not meet as expected. This is a helpful mental image of risk.

Figure 50. Risk of Not Staying Aligned

This mental image helps get across that we’re talking about how to succeed in the face of various risks to the project.

Agile helps reduce the chances of that happening by offering more opportunities to inspect and review our progress. Lean-Agile also provides those opportunities earlier in the process than traditional methods of batching at the lesson level or even batching at the entire course level. By seeing courseware increments that work every two weeks, the customer gets to regularly see for themselves if we’re on track for them. They also get to see how responsive we are to corrections if they feel we are off track.

There may be social structures in the organizations that are hard to modify. If you can join your customer contact in meeting directly to influence other decision makers or influencers, you may be more successful than trying to arm your contact to wage this influence battle alone in their internal structure. If they are still new to Lean-Agile, they will not feel comfortable addressing specific objections that resisters of change may put forward.

If you know the reasons well, and you are persuasive, you can listen for the real drivers of why they don’t want to change and attempt to earn your way into a hearing.

[]13.9. The Urge to Batch Larger to Avoid Many Kanban Cards

Early on we had some people who did not think they would like visual management. They did not like the idea of putting every work item on a card. It seemed a waste of effort to them. They wanted to sub-optimize for their personal ease rather than for the team collaboration.

The virtual Kanban board we were using at the time offered the ability to add lists to each card. So one person made a few cards with large lists on them. They felt they had saved time. This particular supplier’s UI hid the list on the card.

So when the team met at the board each morning, the work was not visible to the entire team. This person had sub optimized for their own cycle time rather than for the entire team. The team could not easily see the bottle necks because there were not visual cues for each work item that this person had put inside a single card. That card did not move to different process step columns for a long time. This “savings” he had tried actually stopped the team from seeing what was happening with his work.

Therefore we suggest avoiding the urge to batch many items in a list on a single card. This is technically using Kanban, but it is not applying Lean principles for smaller batch sizes. It also does not make the work visible as well as the team needs to collaborate easily.

During Retrospectives, the team realized they needed to see the work better. Then they switched to using one Kanban card for one work item. This may have taken a little bit longer to set up initially, but it helped the team the entire duration of the project to better see the progress and the bottlenecks and how to remove any impediments more quickly.

Since then we have noticed another supplier, Trello, shows the list on the card front, which allows the team to see it.

Figure 51. Moving a Trello Work Item Card

Figure 52. Adding the List to the Card

Figure 53. Trello Shows the List on the Card Front so the Team can See

Whether you decide to put one task per card or allow lists on cards, check out Trello.com. Their product can be organized like a Kanban board easily.

[]13.10. Messages about WIP Limits and Inventory Waste Don’t Get Through in a Single Telling

Unless trained and coached in Lean principles, many people new to Agile, and specifically to Lean, do not get work in process (WIP) limits and don’t see inventory as waste in knowledge work.

We have found it helpful to remind them of when their boss gave them too many things to do at once and the whole pile took longer to do. This leads to a discussion about context switching waste, and if they are open to it, they start to understand the theory of WIP limits. Then they get on the Kanban board, and from habit, they take 11 work item cards and drag them into their in process column on the Kanban board.

Figure 54. WIP Limit Indicated at the Top of the Kanban Column

We have found it takes a week or two of noticing cards in WIP columns instead of queue columns and giving reminders to the team before teams new to Lean-Agile get this down. Coaching during daily standup meetings is needed. Telling them once or having them read about the principle once does not get it into practice on a daily basis.

Similarly, knowledge workers don’t seem to see inventory as a waste. This has to be demonstrated with an activity or demonstrated on their Kanban board. You can also use examples of other types of work they may be familiar with, like another department that centralized a particular service and then the entire workflow slowed down for everyone.

When you see the board like this, you have to tell them to put the cards back into the ready queue prior to that process step.

Figure 55. Example of WIP Limits Ignored

These messages need to be repeated like you are marketing chewing gum or some other product until you have mindshare of the team.

[]13.11. Customers Familiar with Pure Scrum for Software Development May have Misconceptions

After reviewing Michael Allen’s Successive Approximation Method, we did not think it went quite far enough for what we needed, so we borrowed from both Scrum and Kanban. Because many training customers have not heard of Lean-Agile before, we decided to stick with generic terms like iteration rather than Scrum-specific terms like sprint.

During one project kickoff meeting, two stakeholders for a particular customer clearly had experience or at least much exposure to the Software industry’s use of Scrum, one method of Agile that is frequently used often in Software development teams. They also were new to learning experience development.

One of the stakeholders started asking questions in the technical jargon of Scrum, leaving many others in the room seeming bewildered. We asked everyone else if we could address her question in the jargon she is familiar with. They agreed, and we explained the differences between how pure Scrum works in software development and how we have merged both Scrum and Kanban and have made adaptations for the learning experience domain.

Eventually, after answering more of her questions, she was okay, and the meeting proceeded. Although this turned out well, a situation like this could have easily derailed our plans to use Lean-Agile for this customer if we could not have answered her questions well right there in the public meeting with all the customer’s stakeholders. Her use of the Scrum jargon had all her stakeholders looking to her and giving her credibility for driving what would happen next.

You have to know Lean-Agile principles well to address the various situations that may come up. Spend the time to build your knowledge, so you can help customers follow your approach.

[]13.12. Sometimes Customers Still Want the Courseware Before It Can Be Completed

We have occasionally experienced customers that are familiar with consuming courseware but don’t really know how long it takes to produce it. This is similar to people reading a book or watching a movie but not realizing how much time and effort went into producing it. In other instances, the customer’s downstream customers demanded the courseware as soon as possible, resulting in a very aggressive schedule.

To help resolve this, we have sometimes increased the development teams. We recommend caution with this. Read the book called “Mythical Man Month.” This is not a press-the-easy-button solution. Adding people adds complexity that adds transaction costs to the team. Here is where Lean-Agile can help and make it more manageable too. The daily synchronization meetings that we call daily standups and that Scrum calls daily scrums really helps reduce time spent by anyone waiting to be told what to do next. The Kanban board with their avatar or picture on their cards aids accountability to the entire team for their work.

You will still have issues like ensuring that the courseware has a single voice, and other challenges, but it is easier to execute such accelerated work with Lean-Agile than without it.

We have had buyers ask for even larger-sized teams so we can bring in the delivery time even more. At some point, you may have to push back. Using the Theory of Constraints, we ask about SME capacity as a potential bottleneck. You can easily overload SMEs and create a SME bottleneck if you make the development team too big. You also increase the risks to the producer if the customer asks to stop development for any reason. If on fixed-price, too large of teams adds risk that you need to mitigate.

[]13.13. Without Learning the Principle, Some Team Members Just Go Through the Motions

Some people think that Lean-Agile is some exact process where you can follow the steps without really getting the principles, and all will be well.

Agile is a set of principles.

Lean is a set of principles.

Applying these principles in various situations requires adaptation to the circumstances. What is shown in this book is how we adapted the principles to our specific situation. We share our story so you can see our adaptation in our context. Your adaptation may need to be different. If you apply the principles well, you’ll be okay.

You can’t just follow exactly what worked for another group without thinking through the principles and expect everything to be fine.

[]13.14. Breakdown Video Production into Smaller Work Items

We recommend breaking down video production into smaller work items because its duration may cross iterations.

Large components of courseware like video may need to be broken down into smaller work items. Instead of having a single card on the Kanban board that says “Video Component”, you may need to add travel, arranging for the capture session, post capture editing, etc.

Sometimes media work requires design and a series of tasks to complete too.

If a customer employee will be in the video and that person is not available for 4 weeks, then you’ve crossed two iterations before the video capture session.

The principle is to break work items down into a size that can be completed in a two-week iteration.

[]14. Successes[]

[]14.1. How to Convince a Customer to Accept a Lean-Agile Approach

Many customers are concerned about

  1. {color:#000;}Cost

  1. {color:#000;}Schedule
  1. {color:#000;}Scope
  1. {color:#000;}Risk – Whether or not your organization can do the work

So you have to address each of these areas.

Lean-Agile speeds development, allowing you to offer a lower cost.

Because Agile Scrumban treats scope rather than time as variable, something will be delivered at the pre-determined milestone – the schedule does not slip during the portions of the project where you apply iterations.

[]14.2. Early Success Helps Gain Momentum

Set up your first team for success. An early win helps your leadership, team and customer buy in to Lean Agile.

[]14.3. Stack the Team

If you have the ability to influence or pick who joins the first attempt at Lean-Agile, we suggest you first try to pick positive people. Those that have shown tendencies towards early adoption of other changes have demonstrated they’re ready for your team.

Nearly every group of humans has its curmudgeons. Spare yourself the hassle of dragging someone along in a change initiative first effort. They will complain, ask why over and over, and they may even attempt to sabotage the effort.

You’re looking for an easy win to test and gain momentum.

Some companies have performance management systems and leadership cultures which do not weed out these type of people as quickly as you may prefer, and you may get stuck with some of them in your pool of picks. Save your energy for the implementation. Stack the team with as many of the flexible attitude, curious, willing to try something new people you can hire, borrow or reassign to this effort.

After the Lean-Agile system has the kinks worked out and you know what works for your organization’s parameters (leadership openness to new ideas, culture, politics, etc.), then you will have to either bring the curmudgeons along or help them find new employment in an environment they may like better. If you are in a leadership role, this last choice is probably better for you and them anyway. If you have no influence or control over the performance management subsystem of your organization, then hold these folks off until last so you can build up the transitional culture as much as possible without their resistance energy slowing things.

Sometimes, these cantankerous folks transition from passive complainers to active snipers looking for ways to stop the new idea. If you have these type of people on your staff, you’ll need to arrange with your next level leadership to underwrite mistakes made in the process of improving for a specific transition period.

Some are not so far down the resistance path and have open minds but do not yet see how your new Lean-Agile approach will help them. They may rightly point out that they don’t have to do A or B when they run things their way. You may have to explain that they have been sub optimizing for the single person’s benefit when we’re looking to optimize for the entire team’s cycle time improvement.

[]14.4. Productivity from Team Rhythm

In our experience, one of the most valuable aspects of Lean-Agile is the team rhythm. Lean calls this takt time. The military calls it operations tempo, shortened to “OPTEMPO.” Some call it cadence.

Like the loud metronome used by a high school band and dance team to coordinate the actions of the many people on the practice field, our Lean-Agile system provides that rhythm to the learning experience development team, and the entire team is more synchronized than we were before Lean-Agile. The system includes a daily standup that provides feedback and product increment goals that tell them how many learning objectives need to be completed each day of the two-week iteration.

One small, three-person Lean-Agile team found the benefit of team rhythm in their success on an aggressively scheduled project. They hit all the milestones on time and the customer was very pleased. This seems to back up a comment by David Snowden, that “Conditions of scarcity often produce more creative results than conditions of abundance.” The time-boxed iteration and the cadence help provide the sense of scarcity and clarity about what faces you in the next two weeks.

For instructional/LX designers with less experience, however, be careful to adjust what scarcity means in terms of time available or it may actually worsen their performance.

Contrast this to another project where the large training project had 15 courses. The project included multiple teams with a total of almost 20 people. The person leading it was also an instructional/LX designer and was frequently pulled into quality checks at the team level and into many meetings at the organizational level. We did not use daily standups and we did not know about Lean visual management using a Kanban board yet. This team had delays in cost reporting feedback because the activity reporting frequency was weekly, but the financial reports were only released monthly. The team members occasionally got confused about what to work on when the lead got pulled away so much. The project spiraled out of control and ended up with quality problems and cost overruns.

Before you mentally place blame on the lead, consider that the system structure did not facilitate her success in leading this effort. Looking back at that experience, we are sure that we could have done better with Lean-Agile. This lead would have benefited greatly from the system we use now. To apply Lean-Agile we use a Scrumban variant of Agile. We still use leads because we may have to work with other disciplines like software engineering, mechanical engineering, etc.

  1. {color:#000;}All know the iteration goal for the next two-week iteration, providing overall clarity for that period. The two-week goal tells them how many learning objectives need to be completed each day of the two-week iteration. This is the metronome effect we have noticed. It is more helpful than micromanagement.

  1. {color:#000;}All know their individual assignments and have accountability to the entire team even if the lead gets pulled away because the Kanban board we use includes avatars attached to each work item card, so everyone knows what everyone is supposed to be doing today. This reduces coordination waste for people getting their assignments. You could say, “Well the MS Project Schedule shows all that.” We have learned that the team-updated information board gets updated faster, and all can see it.
  1. {color:#000;}Traffic jams or work item cards in a particular workflow process step are as easy to see as the traffic jams on the highway are to the traffic helicopter each morning. In our old way, no one could see the status until the lead updated MS Project. Often because of the many demands on the lead, the MS Project file did not get updated for a week. Kanban boards, whether virtual or real, spread the update work to the entire team, making it less burdensome on each person and more regularly updated.
  1. {color:#000;}The Lean principle of pull helps avoid inventory waste because as each person on the team finishes their current work item card, they move it to the next column on the board and pull a new one from the upstream ready/buffer column.
  1. {color:#000;}All know that learning objective 21, for example, has to be completed today
  1. {color:#000;}The lead does not have to spend as much time asking each person for their status so she can update the MS Project file because the status for every work item is visible on the Kanban board.

[]14.5. Set Up the Board for the Team, and They Get It Quickly

When playing a new board game for the first time, like Settlers of Catan, it helps if someone knows how to set it up.

In a similar way, set up the Kanban board for the team on the first project. It helps them see what you mean to move a work item sticky note from one column to the next column.

We keep a list of default columns to quickly set up new boards. This helps to ensure each project has both the right process steps and the ready queue columns between the steps.

[]14.6. Visual Management Sets the Conditions for Kaizen

It is hard to improve something you cannot see.

Lean visual management is a principle used to make knowledge work visible. We use Kanban boards to make the work in process visible to the whole team, as well as anyone that happens to glance at it.

The following figure shows an example process using Trello.com as the tool. It could just as easily be a physical whiteboard with columns and sticky notes.

Figure 56. Example Process on a Trello Kanban

In this short fictional process the steps are:

  1. {color:#000;}Design

  1. {color:#000;}Develop

Add a Backlog or To Do column with everything, here fictional task A, task B, and so on. Then use a column for Design and a column for Development. Note that the example also uses ready queues between process steps to help everyone see bottlenecks more easily.

Kaizen is about continual improvement. When the team sees the board every day for the 15-min daily synchronization meeting, everyone can see the current state of the project and spot bottlenecks as easily as you can tell the highway has a traffic jam while on the way to work.

The visual nature of the Kanban board is its strength. Everyone can see the current status, so everyone has the information needed to recommend improvements. More heads are better than one when it comes to improvements and getting well from project problems.

If you notice no one is making improvement suggestions, coach the team about the expectation.

If the leadership also has access to the board, it may reduce Lean transaction waste for reporting progress. They can see the board any time they want and see the current state without interrupting anyone on the team for an update for report X or Y or Z.

Ideally, you convince them to look at the Kanban board at whatever frequency they want. However if your management has many projects and your set of projects is just a smaller subset of them, they may not change their methods of tracking progress just for you. You may have to frame it as an experiment for them to observe to what degree Kanban provides the status information they need.

This is probably why many in software now talk about their goal to have the entire enterprise be Lean-Agile.

[]14.7. Checklists as Job Aids

Use a checklist for the iteration planning meeting, and a checklist for the retrospective meeting to help all of the development teams know what was expected at this point in our Lean-Agile process. Some people are more enthusiastic about a checklist than others.

[]14.8. Working Product Increments Build Customer Trust in the Team

This is one of the terrific things about the Lean-Agile process that was surprising. Others in the software domain talked about it, but until you see it happening you don’t realize how significant this is.

Each iteration, every two weeks, our team delivered a working courseware increment of a specific number of LOs for customer inspection and review.

The customer began to see on the second iteration that we had fixed the global comments they had provided as feedback during the first iteration. They saw the courseware interactivity functioning correctly, and they saw the look and feel and media treatments all ready for Pilot.

After 5 or 6 iterations, this development team and the customer stakeholders had a positive relationship built on trust earned by repeatedly listening to demo feedback and making the changes by the next iteration. The teams using the Lean-Agile approach have consistently built customer trust faster than the traditional approach teams with their customers, where more of the unknowns and risks remained until closer to the end of the project duration.

Development team morale was better. Most of the customer interactions were better. The customer was better served and was more willing to recommend our team than a team using the other approach.

When you first read about Lean-Agile, this does not stand out much. When you experience it, you see the significant influence it has in all of the customer interactions. This is one of the key benefits to the Lean-Agile approach in our opinion.

[]14.9. Build-In Quality using Checklists

Software development is able to automate testing of code to aid the team.

In learning experience development, there are a few things we can automate, but much of the training product quality assurance content checks are still performed manually by hand. Quality criteria vary by organization, so the following table is not exhaustive, but illustrative.

Table 7. Quality Testing

Manual (by hand)|=~{color:#000;background:rgba(0, 0, 0, 0);}. Automated
Instructional quality criteria Textual content spell check
Accessibility quality criteria (section 508) Some grammar checking of textual content
Security requirements Interactivity software code tests
Meets learning objectives from the CDD SCORM conformance
Assessment and measurement[_ Nothing automated yet_]
Usability of the learner interface[_ Nothing automated yet_]
Functionality of eLearning interactivity or blended components[_ Nothing automated yet_]
Game or simulation mechanics, pacing, balance between engagement and pedagogy[_ Nothing automated yet_]

To help ensure that the customer is getting a consistent quality experience, use content checklists and functionality test cases performed by QA before delivery to the customer.

We suggest implementing checklists for Course Design Documents (CDDs), Prototypes, etc. Consider including a link to the checklist in your definition of done card that sits at the top of the CDD Done column or the Prototype Done column.

We recommend a book called, The Checklist Manifesto: How to Get Things Right, by Atul Gawande, January 4, 2011. Apply it for quality for WIP. If checklists work for Pilots and Medical Doctors, then they can also work for learning experience development.

You also may need to convince your organization’s quality experts to think differently about how to prove we use Lean-Agile the way we say we do. Many of these folks approach the world with a records view and will want written record artifacts for proof. Often, these staff are familiar with traditional tools like meeting minutes and forms, but not Kanban cards. So you may need to convince these folks that the time-date stamp functionality in virtual Kanban cards activity logs are records sufficient to prove that we do what we say. Various virtual Kanban suppliers have a similar functions. Find ways like this to reduce transactional waste and keep your project more Agile and Lean than just saying okay to whatever they require might get you. This is part of your change management process when selling Lean-Agile in your organization.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Many quality specialists have more experience with traditional or waterfall approaches to project management where often the quality checking is all done near the end of the project lifecycle rather than during each process step along the way and leveraging the “definition of done.” Lean-Agile can be a big change to people in the quality field too.|

[]14.10. Scaling to Many Teams Spread Over Many Sites

We have successfully used Lean-Agile with many teams in various sites across the United States of America.

These teams have not all performed their development work at the same time, so each experience with different teams of people has provided opportunities for Kaizen.

One way we run multiple teams all operating at the same time is by having the “lead” for each team join a daily standup for all the leads to identify issues that impacted everyone. In a schedule-oriented organization, this can get overrun by the need for schedule updates instead.

A successful way of updating management for entire projects with multiple courses was to use a presentation slide that graphically depicted the Kanban for the entire project. We had tried to walk them through a project-level virtual Kanban board, but the screen sharing application had a low frame rate and the movements scrolling across the board left and right to answer questions was causing nausea for the most senior leader. So, that application was out the window. Using the known, Power Point, and making one of the slides look like a Kanban was the winning ticket in our environment. The columns were similar, but the cards reflected entire courses without any lower level details. This fast visual picture of where we are now hit the mark of shared understanding.

[]14.11. Adapting to Organizations that Enforce Linear Waterfall Methods

Some organizations will be so set on the traditional waterfall methods that it may seem like antibodies are out to kill off the Lean-Agile effort before it builds any strength.

Succeeding in this environment requires getting the organizational leadership on-board with the idea. Either build a skunk works, protected from normal processes, or ask for a waiver on the first attempt to see what it does as an organization-level experiment.

At the beginning, you may not be successful in getting all that you want. Other times a customer may balk at Lean-Agile even if you won your own leadership support.

So if squeezed into a waterfall method, it is still possible to use some of the Agile and Lean principles even if you cannot do it 100% off the starting blocks. You may have to make adjustments and phase in more Lean-Agile as you have success and increased credibility.

The Lean visual management principle improves team synchronization. We have witnessed that and validated it is an improvement.

If you choose a stealth approach, Kanban does not have to include the customer or even organizational leadership. You can implement it without any change to your existing processes. Then, get it working and show an achieved success.

If there are phases of your process that only involve a single person, Kanban may not add enough value to use it until later in your process where two or more people are coordinating their efforts.

Another area of difficulty is schedules. If you get the okay to attempt Lean-Agile, schedule down to the iterations and stop if possible. Your iterations may include at a minimum:

  1. {color:#000;}Iteration Planning Meeting

  1. {color:#000;}Delivery of iteration goal (list of scope to be done during the iteration)
  1. {color:#000;}Iteration Design/Development
  1. {color:#000;}Iteration Review/Demo with Customer stakeholders
  1. {color:#000;}Iteration Retrospective

If your organization expects the use of historical metrics to estimate the work durations for the schedule, then you may have to use a combination of historical metrics and team estimates, or all historical estimates and no team estimates. Do what you can and prove it is helpful in order to get the credibility to ask for the room to implement more of Lean-Agile on the next courseware project.

Often times customers still expect the traditional deliverables such as:

  1. {color:#000;}Course Design Document

  1. {color:#000;}Prototype
  1. {color:#000;}Storyboards and Scripts
  1. {color:#000;}Working courseware package or instructor support package

You may have to do some explaining about what changes in a Lean-Agile project.

[]14.11.1. Courseware Design Document (CDD) Adaptations for Lean-Agile

You may be able to successfully explain that a CDD is a so-called big requirements up front approach that is inherently waterfall, and you may be able to convince the customer or your leadership to skip this deliverable completely and instead perform the design effort in chunks distributed across the iterations.

Or you may have to create a CDD up front. In which case, you may have to skip fixed time box iterations until the CDD lists all the learning objectives, sequences them, shows the instructional strategies planned and describes the evaluation strategies planned. See the section on Lean-Agile Design for more detail.

Whether or not you can align the schedule into iterations and make the scope negotiable depends much on your current business processes like customer negotiations, contract processes, etc. if you make work for customers to buy. For internal projects, it depends on how well you persuaded the organizational leadership. If possible, get into the contract or the leadership expectations that scope will include the highest priority work items, so that each iteration you are working on their highest priorities.

If unsuccessful at this, you may be trying to hit both fixed schedule with iterations and fixed scope. The customer may expect that all of what they want in the courseware gets included. You can still use prioritization.

If the customer does not agree with the prioritization approach, or fears confusion among their stakeholders, or just wants it another way, they may ask that you deliver courseware increments in the traditional order such as Lesson 1, followed by Lesson 2, followed by Lesson 3, etc. If this happens, then your prioritization is set when you decide at design time which learning objectives go into which lessons. You will still need to help them understand that some work items may get shifted to adjust to changing circumstances like SME availability, for example. As long as they get their way in the rule, then they don’t seem to mind so much that there may be some exceptions.  

Some customers and training leadership are hesitant to give up their traditional up-front blueprint CDD, so you may have to adapt using less Lean-Agile for the CDD and more for other parts of the development process.

Another option is to adapt by including multiple cycles of two or three days each, rather than standard two-week fixed time box iterations, for building a CDD because CDDs typical expectations are often for CDDs to be done in a period of a few weeks. You may need to adjust how much time can be allocated to the CDD, depending on the length of the courseware. The following is an example of what could be included, not a prescription of what you have to do.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. The following design descriptions are illustrative rather than exhaustive.|

Design Cycle 1

  1. {color:#000;}Review the formal training project statement (states the training needed to meet the requirements—What, Why and When).

  1. {color:#000;}Review any products of the Training Needs Analysis (TNA) or other information-gathering results from customer meetings.
  1. {color:#000;}Identify achievable LOs and their grouping and sequencing.
  1. {color:#000;}Identify enabling objectives and key learning points, if applicable.
  1. {color:#000;}Identify any instructional/LX design factors and constraints.
  1. {color:#000;}Collaborate with SMEs.
  1. {color:#000;}Document the design consensus in the CDD (to avoid surprises).
  1. {color:#000;}Conduct synchronous customer review of CDD increment #1 (Purpose, LO scope and sequencing).

Design Cycle 2

  1. {color:#000;}Incorporate customer feedback into CDD increment #1.

  1. {color:#000;}Identify the assessment strategy to constrain what is assessed and how (formative diagnostic progress assessment, type of test, summative assessment of each TO/CTO, associated checklist for reliability and objectivity, pass/fail grading, remedial training strategy, and record keeping).
  1. {color:#000;}Determine the appropriate blend (technology-enhanced learning [mobile, eLearning, immersive virtual simulations, adaptive learning intelligent tutors, etc.], devices[desktop, part-task, full-task, simulators], social, human-facilitated [classroom, virtual classroom).
  1. {color:#000;}Identify delivery options and the design selection rationale.
  1. {color:#000;}Select methods and media (including any constraints that may limit options) for implementing LOs.
  1. {color:#000;}Associate lessons to related groups of LOs.
  1. {color:#000;}Specify the course (course and lesson/event titles, related LOs and dependent key learning points), administrative notes (duration, location and essential references), support expectations (handouts, exercises, equipment) and risk management (assessment and mitigation).
  1. {color:#000;}Estimate delivery duration.
  1. {color:#000;}Collaborate with SMEs.
  1. {color:#000;}Document the design consensus in the CDD (there shouldn’t be surprises)
  1. {color:#000;}Conduct synchronous customer review of CDD increment #2 (assessment strategy and LO instructional strategies).

Design Cycle 3

  1. {color:#000;}Incorporate customer feedback into CDD increment #2.

  1. {color:#000;}Conduct synchronous customer review of the design/CDD.
  1. {color:#000;}Negotiate course scope and implementation features if the contract allows Lean-Agile adaptation instead of waterfall prediction of schedule.
  1. {color:#000;}Incorporate customer feedback (if still required).
  1. {color:#000;}Obtain customer acceptance of CDD (if required).

The point is to use an iterative approach to the courseware high-level design, if you got customer buy-in before starting.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note[_|<{color:#000;background:rgba(0, 0, 0, 0);}. The preceding is still hybrid Lean-Agile. To the degree you can convince customers to go 100% Lean-Agile, you may not need the entire design done up front. _]|

[]14.11.2. Prototype Adaptations for Lean-Agile

Much of how you do the prototype depends on how much Lean-Agile you got accepted. If you were able to successfully negotiate iterative 2–3 prototypes then the first prototype will be a rough quality of finish, getting refined each iteration, as suggested by Michael Allen in his book, Leaving ADDIE for SAM.

However, if the customer insisted on a single prototype that is of high quality finish, then you may not have the option of more than a single iteration on a prototype.

Customers are less likely to misunderstand your intent with a working prototype than with the text-based CDD because they do not have to read your words and form their own mental image. Instead, they get to see your implemented examples of functionality. This idea of reduction of risk of misinterpreting design intent can help you in up-front negotiating.

Make functional prototypes your goal rather than using example content or lesson 1 content. This constraint helps to keep all the stakeholders focused on the delivery package rather than the contents during review at this point in the development life cycle.

If this particular courseware breaks no new ground, so-called “green field,” then the prototype effort may go quickly. If however, you are exploring how to leverage a new technology or some other unproven method, then the prototype may take longer than prototypes for more established methods. If doing exploratory work, prioritize the hardest things first.

[]14.11.3. Storyboards and Scripts

If you were successful at negotiating a small-batch Lean-Agile approach over a large-batch traditional approach, then you may need to reinforce customer and leadership expectations at the storyboard/scripts deliverable. Historically, storyboards have been for large batches like lessons or entire courses.

The key for a more complete implementation of Lean-Agile on this deliverable is to persuade that rather than the traditional method, which is a large batch, of storyboarding the entire course, or even rather than the not similarly-sized (Lean principle) medium batches of storyboarding entire lessons at a time, that Lean-Agile is better served by smaller batches of courseware being storyboarded and scripted.

You may want to explain that the fastest cycle times will occur by working on a single learning objective small batch at a time. That the development team will address LO 1.1, and then LO 1.2, and then LO 1.3. That when the text for LO 1.1 is in the courseware authoring tool then that LO is storyboarded. Then the team will proceed past the storyboarding deliverable and build out the interactions and media for that LO. Be sure the customer understands at the iteration review/demo that we will deliver what we think is a 100% complete LO 1.1 ready for delivery to students/learners (use your customer’s preferred term).

We have found it helpful with many customer stakeholders to use a simple mental image that is easy to create. We use this image of Lean-Agile that shows the work items as miniature cubes in a block of cubes that resembles a Rubik’s Cube toy. But it is a great illustration to help people follow the piece-by-piece approach. We are building the final Rubik’s Cube with each iteration being a layer of the Rubik’s Cube a mini block at a time, and at each iteration the customer reviews the courseware increment or the layer of the Rubik’s Cube.

Figure 57. Rubik’s Cube Image of Agile during Development

Some customers insist that storyboards be text only in the authoring tool, for mobile learning and PC-based eLearning, and for instructor-led training the text-only instructor packages that include the presentation materials, instructor guides and student guides.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{background:rgba(0, 0, 0, 0);}.
p<{color:#000;}. In our experience some customer stakeholders have not mentally adjusted to the speed increase in cycle time that happens by addressing a single LO at a time.

Early on in our Lean-Agile implementations, we expected the customer to look at the storyboards for each LO. If your scope is, say, 11 LOs in a single a two-week iteration, this means that the customer would have to look at 11 smaller storyboards. In real life, many do not. Instead they wait until the iteration demo because that is on their schedule. Even if invited to your Kanban board, they may not take the time to look daily because they typically have other jobs they have to do for their employer. This led to asking various customer stakeholders to consider their storyboard deliverable as met when the current ‘state’ of the authoring tool had all the text included.

For those that required a storyboard deliverable, this could meet their process requirements. Even if no review is conducted on a work item, the iteration demo may catch it. Although this kind of mistake could have been disastrous in a ‘large batch’ approach, the iteration courseware increment was a smaller batch of up to 11 LOs, so the degree of rework if any was minimized. Other work arounds are that the instructional/LX designer is collaborating with the customer’s SMEs daily while developing the ideas for how to implement each LO, so the iteration review was normally not a surprise to the SMEs at least.

The customer leadership needs to understand up front that the entire deliverable for storyboards/scripts will not be complete until all the increments are delivered. This adaptation can allow your team to still apply Lean-Agile and to still interface with others who are still mostly in the traditional training approach of large batches.

Note*][_ Another technology consideration is to use plain text files for storyboards. The developers can easily convert plain text to markdown or asciidoc, which is easily converted to markup. Plain text keeps the content developers focused on the content goodness and not on the layout and formatting. The ease of conversion from markdown text to markup in HTML5 or XML is exceptionally cost effective. For teams using rapid development tools, you’ll be stuck slowly entering content into form-like entry fields and markdown will not help you until the suppliers of those tools add a markdown text import method. There are very few products on the market that include semantic encoding of training product content._]

[]14.11.4. Completed Lessons Deliverable Adaptations

For customers that require a specific deliverable that is some reasonable variation of completed, and working lessons, this may include fixing any feedback provided during the iteration demo.

By tracking change requests (CRs) or defects as Kanban work item cards, the entire team can literally see the scope of the technical debt incurred during iteration 1, so when iteration 2 is planned, they can adjust their scope to include finishing incorporating these changes into increment 1, and then working on iteration increment 2. As soon as we deliver increment 1 with all changes incorporated, then that meets the completed lessons deliverable for that increment.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. This is where we’ve had the most push back from teams, so you may need to find a better solution here. Some people get frustrated adding a Kanban card for every customer change request, thinking it “wastes time.” You may need to explain the value or find another way to make this change demand visible to apply the Lean principle. The customer leadership needs to understand up front that the entire deliverable for completed lessons will not be complete until all the increments are delivered. This adaptation can allow your team to still apply Lean-Agile and to still interface with others who are still mostly using the traditional training approach of large batches.|

Takeaway: Do as much Lean-Agile as you can by degrees and avoid purist, all or nothing, binary approaches. Make your motto, “Data over dogma” as you experiment with what works for you.

[]14.12. Many Heads are Better Than One – Innovation by Involving the Whole Team

In traditional project management approaches, the leader has the tools to see the entire project. Often, a spreadsheet is the tool used. Sometimes, an MS Project Gantt chart is used. Other times, MS PowerPoint slides that report progress show the view.

The common theme on these approaches is that the file is on a computer accessible by the leader only and not the team.

Using Lean visual management can be easily done using Kanban. There are other ways too. Kanban makes the current situation of the team’s progress visible to all.

When the entire team can see who owns each work item that is in process (not in backlog or done), then it becomes easy to see when there are problems.

All drivers can see the traffic jam on the LA freeway. All can start looking for workarounds. Similarly the visual management mechanism you choose, we use Kanban, helps all team members see what is going on. During the daily synchronization meeting, the team looks for bottle necks. Then following Goldratt’s Theory of Constraints, they can address the slowest bottleneck of them all first. When that one is improved, they can move on to the next bottle neck.

Everybody can offer ideas when everyone can see the problem on a visual management system. When the progress is only visible to one person who can access the file format that contains the progress status, then only that person’s head can help solve the situation. If all the team members get this much feedback:

  1. {color:#000;}See the problem

  1. {color:#000;}Be involved in the development of an intended solution
  1. {color:#000;}See the solution experiment (the impact of the attempt to solve it)
  1. {color:#000;}See the consequences – good if the problem was solved, and the next experiment if not

Then you have both a learning experience development system and a problem-solving training school for all the team members involved. This spreads your problem solving expertise across more people as the data educates them on what works best and what does not work in various real-world circumstances. This learn-by-doing grows the entire team.

The team may see that many Kanban cards have piled up waiting for the ‘Test’ step, a quality assurance check against a particular checklist of test cases. They may need to stop pulling upstream work item cards and more of them pull cards in the Test step to help those work items cards get completed. You may need to reinforce the Lean one-piece flow principle with the team, and that we get zero credit for incomplete work items. If using earned value, the 0/100 technique applies. As you coach people away from sub optimization at the individual level and towards optimizing for entire-team throughput, your cycle times will increase even if your individual team member utilization is not as high. This is okay in Lean-Agile. You will not end up with slackers because they can pull another work item card. You will end up increasing your cycle time for work items, and you will have more completed, working courseware to show earlier in the timeline than in a comparable waterfall approach.

The Lean-Agile coach may ask why Bhodi pulled 15 work item cards into the WIP lane, showing his ownership when, he as one person, can only realistically work on 1–2 work items at a time. Then you have a team teaching moment for all to realize that Lean pull is different, and that they are expected to leave most work items in the buffer queue until they are actually hands-on working on a particular work item.

The lead may notice that Dave’s cards sit in process step columns longer than expected. He can ask clarifying questions. If it turns out that Dave cannot keep up with the rest of the team, Dave may have to be replaced.

[]14.13. Adapting Lean-Agile to Fit Within CMMI

This section addresses Capability Maturity Model Integration (CMMI) for development.

CMMI is a process improvement training and appraisal program and service administered and marketed by Carnegie Mellon University (CMU) and required by many DoD and U.S. Government contracts.[70]

It is often applied to software projects. Training professionals may not have heard of it or may not have had to comply with it depending on the way their organization approaches the development of products and their customer’s requirements. Training groups that support engineering organizations making complex products, especially for the U.S. Government, may be more likely to have exposure to and have to comply with CMMI. It is similar to Six Sigma and ISO 9001 in many ways but different enough that you’ll need further reference material to apply it.

Briefly, it represents the following process areas and lists goals and practices for each.

Maturity Level 2 – Managed [70]

  1. {color:#000;}CM – Configuration Management
  1. {color:#000;}MA – Measurement and Analysis
  1. {color:#000;}PPQA – Process and Product Quality Assurance
  1. {color:#000;}REQM – Requirements Management
  1. {color:#000;}SAM – Supplier Agreement Management
  1. {color:#000;}SD – Service Delivery
  1. {color:#000;}WMC – Work Monitoring and Control
  1. {color:#000;}WP – Work Planning

Maturity Level 3 – Defined[70]

  1. {color:#000;}CAM – Capacity and Availability Management
  1. {color:#000;}DAR – Decision Analysis and Resolution
  1. {color:#000;}IRP – Incident Resolution and Prevention
  1. {color:#000;}IWM – Integrated Work Managements
  1. {color:#000;}OPD – Organizational Process Definition
  1. {color:#000;}OPF – Organizational Process Focus
  1. {color:#000;}OT – Organizational Training
  1. {color:#000;}RSKM – Risk Management
  1. {color:#000;}SCON – Service Continuity
  1. {color:#000;}SSD – Service System Development
  1. {color:#000;}SST – Service System Transition
  1. {color:#000;}STSM – Strategic Service Management

Maturity Level 4 – Quantitatively Managed[70]

  1. {color:#000;}OPP – Organizational Process Performance
  1. {color:#000;}QWM – Quantitative Work Management

Maturity Level 5 – Optimizing[70]

  1. {color:#000;}CAR – Causal Analysis and Resolution
  1. {color:#000;}OPM – Organizational Performance Management

Optimization is built-in to the Lean-Agile approach. We have explicit process step organizational policies such as the definition of done for each Kanban column. We have people doing their work against a defined process (the steps across the board). We have the data collection with cards moving across the Kanban board make the state of the total work in-process visible, therefore measured. With some virtual Kanban systems you also get time-stamp data on each card movement, providing additional process data.

Lean-Agile makes optimization based on the data collected easier to execute, allowing teams to move all the way to CMMI level 4 and 5.

Risk mitigation steps can be added to the Kanban board.

Agile can help make CMMI more doable with less paperwork overhead. Artifacts are still generated, but are built into the Agile methods you implement to meet the Agile and Lean principles.

There is too much scope within CMMI to go into it all in this book, so instead we refer you to CMMI Second Edition, by Mary Chrissis, Mike Konrad, and Sandy Schrum.

[]14.14. Pull Versus Assigning (Push) Works Better

Teams assigned to Lean-Agile for the first time bring with them the habits of traditional approaches to courseware project management. The Lean-Agile Coach had to remind teams and Leads to use pull tasks rather than assigning (push) work as the rule, then doing any exceptions. It was different from prior experience and for some took longer to abandon than others.

This next point is repeated from earlier because it is important in shifting to the Lean-Agile mindset. New teams would have someone pull 15 or 20 work item cards into their work in process column on the Kanban board. At the next daily synchronization meeting, the coach would ask why. “Those are the things I have to work on” would come the reply.

Then, the Lean-Agile Coach would explain that the metrics behind the scenes with the virtual Kanban board is measuring hands-on time versus waiting-in-a-queue time. Hardware maintenance people sometimes call this “wrench time” versus other. We would explain that we want 13-14 cards to stay in the ready queue just prior to the WIP column and for the person to pull only 1-2 cards into the WIP column.

Some would voice agreement but take a few days to get it right. Others got it faster.

Traditional process steps might look like this:

  1. {color:#000;}Process Step X

  1. {color:#000;}Process Step Y

The person working process step X finishes his work items and pushes them to process step Y. If the person in process step Y cannot keep up then a bottleneck builds.

In Agile, most process stations, columns on the Kanban board, in the process flow include at least two columns:

  1. {color:#000;}Ready queue (typically the prior process step A “done” column)

  1. {color:#000;}Process Step B WIP column
  1. {color:#000;}Process Step B “done” column
  1. {color:#000;}Process Step C WIP column
  1. {color:#000;}Process Step C “done” column

The person finishing step B moves their work item card as they complete each into their own Step B “done” column and pull another card from step A “done” column which is their ready queue. This is like the stop light on the LA freeway on-ramp. The person working Process Step C, pulls only 1-2 cards at a time as they have open capacity from Step B “done” column. They work each card and move it to the Step C “done” column.

Kanban applies Goldratt’s Theory of Constraints. The summary of this theory is:

  1. {color:#000;}The whole system performs only as well as its bottleneck.

  1. {color:#000;}Ensure you have slack before and after the bottleneck. This happens automatically with pull scheduling.
  1. {color:#000;}Use the slack to fix the bottleneck.

Pull prevents bottlenecks.

A queue distinguishes work that is eligible to be pulled from work that is still in process. Queues start backing up immediately following any blockage.

When someone needs another task to perform, they pull the top item from their ready queue or the backlog because that is the highest priority work item.

Daily synchronization meetings with the team traverse the Kanban board from right to left to emphasize pull.

Pull is work processing actually triggered by downstream customer demand rather than a forecasted schedule, and Kanban signals visually trigger this demand to flow through the value stream. As the Kanban columns/process steps empty out, team members have a visual signal to pull another work item card from the prioritized backlog. Kanban column WIP limits indicate the maximum capacity for that process step and the maximum work items the team member should pull into this column. Although you may see the Lean manufacturing roots in this definition, it applies equally well to knowledge work, so we use it.

[]14.15. Skip Kanban Columns When Needed

The principle is visualize work flow.

The columns of a Kanban board appear linear because of the 2 dimensions of a typical whiteboard and the virtual tools that mimic physical whiteboards.

However, we also use optional columns. This is our effort to reflect the network flow of work over linear flow. We still need visualization. Skipping columns as needed helps get there as a simpler first step. As always, our continual improvement focus is on optimizing for the flow through the entire process. People on our teams have adjusted to skipping columns as needed pretty well.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. For learning simulation projects, we use a network of Kanban boards to reflect the nature of the work flows better. Although that is also possible for all Kanbans, this gets down to a leadership judgment about what teams new to Lean-Agile can handle initially, in our opinion. We suggest that networked Kanbans be delayed until your organization has familiarity with basic Kanban boards first. Judge when or if to introduce it based on the Lean-Agile maturity of your organization. Experiment first, as always. Share what you learn.|

The following figure shows the best conceptualization we have found to date on networking Kanban boards.

Figure 58. Jurgen Appelo’s Sketch of a Possible Network of Kanbans [71]

For example, some teams prefer to use “blocker” indicators on their Kanban cards using some visual signal like a red sticker added to it. Virtual Kanban providers include a variety of ways to accomplish this too. We have found their various ways tedious and too time consuming, so we just added an extra column.

For example, if the process steps were

  1. {color:#000;}Step A

  1. {color:#000;}Step B

Then we add the column that most often blocks us and put a WIP limit of two on that column. For example:

  1. {color:#000;}Step A

  1. {color:#000;}Waiting on SMEs
  1. {color:#000;}Step B

Team members are not having issues with SMEs like delays, for example, then they move their work item card from step A to step B directly. However if someone is hitting a SME blocker, then their card stops in the Waiting on SMEs column and the team and the project leadership all have a visible signal that something is slowing down the progress.

For other situations that do not come up as frequently, we use the normal red sticky note or the blocker functionality of the virtual Kanban board, and explain the reason for the blocker briefly.

We don’t mean to pick on SMEs, it is just that this often occurs in learning experience development projects, so it is a familiar example to our readers. Many SMEs adjust to the faster pacing and report enjoying Lean-Agile because they don’t get a HUGE batch of work to review all at once, but that instead, we ask them to review smaller batches or bites more often.

[]14.16. Iteration Demos Unify the SMEs

We have performed some learning experience development projects where the customer assigned many SMEs and did not plan any means of arbitration when two SMEs had strongly held and mutually exclusive opinions to provide to us as change requests. This situation is difficult for both the customer stakeholders and the development team, and it slows throughput.

One of the extra benefits of iteration demos that include all the applicable SMEs as attendees and with the customer leadership representative invited is that the SMEs tend to more easily work out any differences. The customer leader hears the issues and directs an offline solution after the demo with a due date. Or perhaps the SMEs involved hear each other out and realize they are more aligned than originally thought. Or sometimes, the customer makes a decision about which input to accept and which to not implement. Whatever the means for getting there, the demo provides the visibility into the issue at a level of observation of all the relevant stakeholders. It get resolved by the customer much more quickly this way than with endless emailed arguments back and forth. This is especially true when the iteration demo has a fixed time box and all are aware of the limited time to cover the major issues during the demo and to see working courseware for the increment delivered.

[]14.17. Training Already has a Way of Decomposing a Work Effort and Estimating

Training development projects historically have tended to apply the following scalar or product breakdown structure:

  1. {color:#000;}Curriculum/Learning Path

  1. {color:#000;}Course/Learning Experience
  1. {color:#000;}Lesson/Learning Experience
  1. {color:#000;}Learning Objective/Performance goal
  1. {color:#000;}Teaching Point/Key Learning Point

Because we spend much time persuading professionals in the training domain, rather than decision makers with a primary proficiency in the software domain, we made a design choice with our approach to Lean-Agile to blend as much from their familiar training domain as possible.

Software is more varied and abstract and the product breakdown structure varies more often depending on the intended functionality. So the software Agile literature proposes many ways to decompose a project into similarly sized chunks.

[]14.17.1. An Aside on Estimating Relative Sizes

Often, project leadership receives flawed estimates when they ask a development team to estimate in hours. To avoid that outcome, many Agile groups use relative estimation to estimate their work effort. This method is very different from estimating the exact or absolute effort using hours as the unit of measure.

Let’s be clear, nothing in Agile requires using relative estimation techniques. However, relative estimation has become the most popular method in Agile. We can use any desired estimation method with Agile. With that caveat out of the way, here is why many Agile practitioners use relative estimates to predict their own work effort.


  1. {color:#000;}Estimation is hard.
  1. {color:#000;}We need estimates to set goals for scope, schedule and budget to track progress towards the desired results.
  1. {color:#000;}We need estimates to make trade-off decisions.
  1. {color:#000;}We want fast estimates. Estimation is time consuming, with many absolute estimates involving substantial rework during execution. Lean calls this waste. Relative estimates involve less waste.
  1. {color:#000;}We need an effective technique for estimating emergent requirements. Scope often changes.
  1. {color:#000;}Complex projects have many moving pieces with hard-to-quantify dependencies.
  1. {color:#000;}Humans are better attuned to pattern recognition, comparing two things in relation to each other. People tend to more accurately estimate relative sizes.
  1. {color:#000;}People tend not to be good at correctly estimating with absolute values, like hours.
  1. {color:#000;}Optimists tend to underestimate. Pessimists tend to overestimate. [72]
  1. {color:#000;}“Comparing is much quicker and more accurate than deconstructing.” [72] By deconstructing, we mean to break down a work item into constituent tasks.
  1. {color:#000;}People agree faster on relative estimates like “that is twice the size of the other one”. If required to use absolute values, they may spend more time arguing over whether it was 15.75 hours, 16.25 hours or 17.5 hours.
  1. {color:#000;}In golf, at the tee box, we don’t estimate 210 yards and 7 inches. At that early point in the process we only estimate in yards. Only later at the putting green do we estimate in feet or inches.
  1. {color:#000;}At the beginning of the project, we are estimating with the least information we will ever have in the project. Estimates for six or more months out tend not to be accurate.
  1. {color:#000;}No matter how much effort we put into an estimate, an estimate is still an estimate.

So rather than absolute values, Agile practitioners in the software discipline have largely adopted relative values instead, using various methods. Because this is so different from traditional hourly estimates, let’s discuss how this is often done.


  1. {color:#000;}The most popular relative size value sets in software development is called story points, where we size work items, or user stories, using a variant of Fibonacci numbers as the unit of measure.
  1. {color:#000;}Another popular relative sizing value set in software development is T-shirt sizes, such as XX Large, X Large, Large, Medium, Small.
  1. {color:#000;}Relative estimation is performed at the product backlog level rather than at the iteration backlog level.
  1. {color:#000;}Iteration-level estimation can be estimated in hours like traditional projects because a single iteration is close in time, typically days not months.
  1. {color:#000;}Multiple experts estimate at the same time (The Delphi Technique), like judges at Olympic Ice Skating events.
  1. {color:#000;}Estimate outliers are verbally justified by the expert making the assertion.
  1. {color:#000;}Averaging individual expert estimates lead to better estimates.
  1. {color:#000;}Planning Poker®.

If we will not use hours as the unit of measure, then we need a substitute. The Agile community in the software discipline uses size “points” to size the effort for each work item. Absolute estimates use hours, numbered using a scale of 1–100, with a granularity of one. Relative estimates use a points scale that is split up into a fixed set of 10–12 “buckets” when using modified Fibonacci. Relative estimation intentionally uses a rougher granularity with almost ten times fewer choices for the estimators.

The Fibonacci sequence provides a pattern for sizing things, whether effort or physical objects in nature. The real Fibonacci sequence is named after Italian mathematician Fibonacci. Applications of the Fibonacci sequence appear in nature, such as the relative size between the whorls in a nautilus shell, plants, storms, and galaxies. Fibonacci scaling is fascinating in nature, but beyond the scope of this book. See wikipedia for more information (https://en.wikipedia.org/wiki/Fibonacci_number).

Figure 59. Fibonacci in Nature [73]

Note that actual Fibonacci numbers are 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144 and so on.

Many in the software development community use a pseudo Fibonacci sequence to relatively estimate a work effort. Note how easily we can distinguish between a 3-point sized effort and a 20-point sized effort.

So to facilitate group estimation, Mountain Goat Software invented a game-like estimating activity and called it Planning Poker®.[74] With Planning Poker®, each team member estimates a particular work item relative to a “normal” work item using a pseudo Fibonacci number. Like in the game rock, paper, scissors, all players should reveal their choice at the same time to temper peer influence. To help team members each reveal their estimate at the same time, Mountain Goat Software developed a card deck for Planning Poker®.

Modified or pseudo Fibonacci numbers use a similar size progression on playing cards that all team members can use to relatively size the effort from the work item up. Mountain Goat Software has popularized Planning Poker® with their cards which include: 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100, and infinity.

Figure 60. Planning Poker® Cards by Mountain Goat Software [75]

Why Modified Fibonacci?

Many teams do not cling to true Fibonacci because in the upper regions “34”, “55”, and “89” imply a certainty that simply doesn’t exist [in estimating]. Stating “40” instead of 34 or “100” instead of 55 transports a coarse-grained granularity that reflects the uncertainty in these regions. Simply speaking, the team tells their Product Owner by showing “40” that “this item is too big and has to be refined” while “100” basically means “this will never fit into a Sprint [an iteration], more refinement is needed”.[76]

— Dominik Maximini

The key point here is that the team’s relative estimates are bottom-up from each team member, and they are relative to a normal work item of, let’s say 8-points. This point estimation may be confusing on first exposure. Because the team doing the work will estimate the points, their estimate is tailored to their team.

For more visually inclined readers, the following figure uses colored blocks to see the size in comparison to the other size buckets. Size 8 is in orange because it is the comparison size for relative sizing against all the other sizes shown in blue blocks.

Figure 61. Estimating Relative Size with Size Points

The smaller size points indicate lower uncertainty/risk and higher reliability/precision. Smaller size points also tend to indicate more confidence by the team. While the higher numbers indicate higher uncertainty/risk and lower reliability/precision. Higher size points also tend to indicate less confidence by the team.

[]14.17.2. Back to Training Scalars

When coding is significant in a courseware product development project, for example in custom simulation development, story point estimation methods from the software development domain are used too.

However, if the courseware does not include significant custom code, then we have found that many training professionals prefer to use a scalar or product breakdown structure they are already familiar with, so we have successfully used:

  1. {color:#000;}Course

  1. {color:#000;}Lesson
  1. {color:#000;}Learning Objective (LO)

We have not felt the need to go as granular as teaching points; although, that is possible. For Agile, we have also found is harder to apply to analysis projects, the A in ADDIE, that deal with curriculum design because we typically staff those with a single instructional/LX designer, making many of the Agile concepts more overhead than helpful with a one-person team. We have had success applying LOs as the good enough or effective granularity or small-batch size for purposes of persuading, negotiating, and implementing learning experience development projects. LOs also vary in size, but typically they are more similar to other LOs than lessons are similar in size to each other. Using LOs allows us to meet the Lean principle for even, continuous flow through the system by using similarly sized work items.

Estimating in some organizations is based on historical data of similar projects. Learning experience development that is estimated by traditional student seat hours may be difficult to switch all at once to another method like T-shirt sizes or Story Points. Training groups may not be used to estimating each work item card from the bottom-up. Initially, you may need to continue estimating by student seat hours and then gather work item card data to provide a basis for future estimates for future projects.

Or you may be okay trying to use Planning Poker® cards using story points.

[]14.18. Find a Virtual Kanban Tool that Allows Batching Card Moves

When you first convert people used to tracking their own work, optimized for individual performance, to Lean-Agile, and they have to use a system like Kanban that is optimized for team performance, you may experience some resistance to change.

If your Kanban board is sticky notes with a whiteboard, people are used to dealing with physical objects like sticky notes.

If, however, you use a virtual Kanban board as we do with geographically dispersed teams, then software functionality varies and some people use those difficulties to resist the new method.

We suggest finding a virtual Kanban tool that allows moving an entire batch of work item cards in one action. For example, we have a buffer queue called “Ready for Iteration Demo” where all the work item cards go when “done”. Then the next column is “Demo Done,” which provides a ready queue for the next WIP column called “Fix CRs/Defects”. The first time someone had to move all those virtual sticky notes one-by-one, they started complaining for all to hear. You don’t really need that on a new project that is still trying to build credibility in an organization that is not used to Lean-Agile yet. Even team members supportive of the new move to Lean-Agile will tire of moving one at a time especially if there is latency in the interaction. The transaction costs may drive them to frustration too. It is not much time, but when repeated, it adds up. Psychologically, it is like waiting for an elevator. It may not take long, but it seems longer.

So, find a Kanban tool that allows selecting all the cards in the “Ready for Iteration Demo” column and moving them all at once to the next ready queue, called “Demo Done”. This provides time-stamp data for the process metrics and does not impose a high burden on team members.

Our current two favorite tools are SwiftKanban and Trello. Both allow moving all of the cards from column (SwiftKanban) or list(Trello) A to column/list B, reducing transaction waste. This is a Lean principle.

[]14.19. Planning Staffing Capacity – Matching Supply and Demand

How efficient your staffing has to be depends significantly on how well you addressed this area in the contract with the customer.

Many organizations that hire full time staff make decisions once per year on whether their headcount is too many, just right, or too few. It is like the Goldilocks story but without the happy ending if too many, leading to layoffs, reductions, destaffing, or whatever your organization chooses to call it.

Organizations that hire temporary staff sometimes have a core of full time staff which they supplement with temporary staff for surges in customer demand.

If serving an external customer with a contractual agreement between the two parties, you need to pay attention to the terms and conditions that address when the customer authorizes spending to begin. If you staff before this time, you may have to eat the costs. If you staff after this time, the customer gets frustrated with your responsiveness to their courseware needs.

Leave a buffer to handle variability where possible.

Understand all the relevant parameters for staffing.

[]14.20. Start Agile and Become Lean

Agile principles are few and the ideas are easier to adopt than Lean. You can start applying Agile Kanban without any change to your processes starting today. You don’t need organizational approval to use sticky notes, already in your office supplies budget, and a white board. If you don’t have a whiteboard, use a big window or buy a whiteboard wall adhesive, or buy an actual whiteboard or cork board. The deployment is simple and easy without many approvals needed. Scrum techniques of Agile can be added to your Kanban approach later if desired.

Lean principles are more involved. There are more of them, and how the principles work together dynamically may not be understood as well initially. After you experience some success with Agile, you may have more credibility to spend on convincing higher-level management that Lean is a good idea too. Lean sometimes scares employees who think that the leaning you’ll be doing is to headcount. Headcount reduction is not the aim of Lean. Deployment for Lean takes some degree of training followed by having your Lean-Agile coaches also coach Lean principle applications too.

Be sure you are clear that the Lean needed for courseware product development is Lean for product development, rather than Lean for manufacturing. Toyota has had great success with Lean for manufacturing, but you don’t need visual signals in a bin of parts, you need visual signals for knowledge work process steps like David Andersen pioneered.

[]14.21. Kanban and Daily Standups Provide Accountability Quickly

Some employees don’t like the word accountability. In our experience, confident employees with strong skill levels tend not to mind accountability.

If monitoring a team of brick layers adding masonry siding to a house under construction, you could drive by the site and see the progress. You can see how many bricks are on the wall, how much scrap is laying about, and how many bricks are left on pallets to be done yet.

In knowledge work, you can walk by the cubes, offices or team work areas and see people typing at their computers. Where the work effort is in process is not visible just standing there. You may interrupt the team periodically and ask for a progress report.

Kanban makes the work in process visible down to what step in the process each part is in, how much WIP inventory exists and scrap or defect volume, just like the brick layers’ site.

Because each person claims cards they pull with their avatar or initials, who is doing what becomes visible too. Whether the workload is balanced or not becomes apparent.

Let us tell you about Dave. His name was changed to protect him. Dave joined a learning experience development team of strong performers in a specialist media role. The daily synchronization meetings kept showing bottlenecks at Dave’s process steps. It caused further investigating by the lead. Dave was not keeping up. His speed of servicing work item cards was not keeping up with expectations of X number of LOs per day. He did not ask for help, but he did not perform. He was right out of school and was not used to a fast-paced cadence or tempo like that yet. We coached and counseled him on expectations and gave him time to show improvement. No improvement came. Finally, we removed Dave from the team and replaced him with another person. She worked through the bottleneck and caught up. We could have added additional staff except that this particular project was a firm, fixed-price project, so we had to replace him. He was not fired, but was moved to a traditional, non-Agile project where they trained him some more on improving his personal work throughput.

The Kanban is not intended as a punishment tool. It simply makes visible what is happening and not happening and lets the team decide where they can make adjustments for improvement.

[]14.22. Learning From Each Other

After you have a number of people skilled in using Lean-Agile, they can become a network of internal consultants to each other. This works better than formal “Train the Trainer” programs because they have experience that is recent and relevant and can provide Just-In-Time advice as needed.

[]14.23. Share Lessons Learned

Get all the lessons learned into a repository or wiki that all development team members can reference and ideally add to. This does not have to be a complex software solution. Post text files which are named well. Inputs should be flowing into this repository or wiki following every iteration retrospective. It is okay if the structure emerges.

This is especially important if you use temporary project teams like Hollywood that disband after the project to form new teams for new projects. This provides a way to institutionalize lessons learned for follow-on workers.

Another way that is not document-centric is to have periodic sharing sessions between teams. For example if running three projects simultaneously, have the instructional/LX designers meet together for an hour and talk about what works and what has not worked so well. Do the same thing with media staff and software developers on simulation projects.

Some people prefer the written form of lessons learned. Others prefer the social interaction. Pick what works best for your circumstances and people.

Also update your templates for various deliverables with lessons learned from each project, each iteration and each customer stakeholder interaction.

[]14.24. Share Successes with Stakeholders and your Organizational Leadership

You spent some credibility getting to this point with your buyer or organizational leadership. Show the return on trust, a newly coined term like return on investment.

Ask teams to capture any positive quotes from interactions with customer stakeholders. Iteration demos are a great place for this. A side benefit is that development team members get direct feedback that traditionally they have not heard, except perhaps second hand. Team members really like to hear good things about all their hard work.

Summarize some of the choicest comments for sharing with leadership. Some may be good enough to go into customer testimonials for bidding your next project.

Third party feedback like this is more believable to leadership than your own talking points about your project. If your organization is still hierarchical, your next level leadership may share these quotes with the next level of leadership. If your organization has already flattened its structure, then the level of leadership hearing it first may be the VP level, and they may share that amongst each other. Some of this depends on the culture of collegiality versus competition among executive leaders.

[]14.25. Persuade Organizational Leadership to Underwrite Experimentation Initially

Asking for any organizational process change can be difficult. If you already have the call, then make it. If you have to get higher level approval, then reduce the risks for them.

Ask for the freedom to make a few small, inexpensive mistakes in order to bring a potential innovation to the organization that could pay for itself.

If you get carte blanche, then still only bite off what you can show success for in a brief period of time.

  1. {color:#000;}Project A: Duration 16 months

  1. {color:#000;}Project B – Duration 6 weeks

Pick the shorter project to experiment upon. Be sure the contractual terms and conditions allow for it or that you can negotiate new expectations still if contracts are not executed yet.

Manage expectations as low as feasible. Innovation is messy at first. You need some backup to allow you to fend off the existing process antibodies and to make some mistakes and learn what works in your circumstances amongst the Executives.

[]14.26. Track Risk Mitigation and Opportunity Capture Actions as Task Level Cards on the Kanban Board

Some organizations formally track risks and opportunities in projects.

Although there are dedicated risk and opportunity software tools for sale, our preference is to put actions to mitigate risks and to capture opportunities as task level work item cards directly onto the Kanban board.

You can put due dates on the cards. Some tools like Trello also provide a calendar view. You may want to use a special color for these tasks or just use yellow sticky notes for all tasks including this type. Some virtual board services allow labels that let you filter cards based on their label. This becomes more important when you have a few hundred work item cards on your board. Not only does this provide a way to track these important tasks to closure, but it also provides additional historical data that can help estimate future projects.

[]14.27. Build Lean-Agile into your Process for Getting New Business

If you are only building courseware for your own organization, internally, feel free to skip this section.

If you create training products for selling to external customers, then in addition to using this approach in the actual development work, consider using Agile iterations and more in your new business processes. In some companies new business is handled by different organizations.

At the least be sure to include strong descriptions of the benefits in your proposals to prospects. Help set the stage for using Lean-Agile in the actual development by having it in the entire conversation with the customer.

[]14.28. A Few Metrics Help, Too Many Hinder

Some organizations have no metrics. Going to that pole does not work well. Some have far too many to provide any focus, and these many metrics become an end unto themselves. Going to that pole does not work well either. Seek a balance by using a workable number of metrics, from the perspective of the individual contributors.

Pick the few metrics that matter to improving your organization for now. This iteration of improvement, if you will. Use those metrics to see if you are getting closer to or farther from your objective. When you met that improvement, like after removing a particular bottleneck, move on to the next improvement and consider what metrics are needed to drive improvement. Importantly, drop the now unneeded metrics. We have seen some organizations keep all the old metrics and have asked people to hit as many as 37 separate metrics.

This is not the crucial few.

Apply the Pareto 80/20 principle to metric selection. Pick the 20% of metrics that will get you 80% of your improvements.

[]14.29. Inspect What you Expect

The old adage, “Inspect what you expect,” can also apply to learning experience development, particularly when you are moving to a new approach.

Have your Lean-Agile coach pop in on daily synchronization meetings with teams. Look at the Kanban boards to see if they are being used appropriately. Attend iteration demos. Provide feedback where needed to help the teams adjust to the new approach.

Like any change management effort, saying something about a new approach once will not get humans to make the change. Follow through, repeat as needed, inspect or spot check to ensure that results are coming along as anticipated, and adjust as needed. The PDCA cycle works for this too.

[]14.30. Reduce Process Complexity Wherever Appropriate

Balance process documentation and fast performance.

Some customers require CMMI, and those organizations have tended towards heavy process documentation with audits that confirm the entire organization is complying with CMMI. If considering how far to go for CMMI, such as level 2, 3, 4, or 5, consider how Lean-Agile helps get to the optimization level. Our Lean-Agile uses the data from the visual board and the team’s experience over the last iteration to reflect on what worked and what needed improvement. The team has data on the visual Kanban board to help aid their decision making. This starts to lean towards levels 4-5 of CMMI.

[]14.31. Standardizing Lean-Agile is Not Antithetical

Some people that moved enthusiastically to Lean-Agile pushed back when we standardized our approach. In their own research of Agile, those folks had picked up a sense that Agile was a free and open way of work and that standardization was antithetical to saying we are Agile.

We combine Agile and Lean, however. Lean also includes the idea of standard work.

Another side effect of standardizing internal tasks is that it reduces complexity. You see this on shows where someone comes in to save a failing restaurant. They reduce the menu and focus on fewer, better options.

This is what we are doing with Lean-Agile. We are optimizing for the entire organization’s performance, not just the single development team’s performance. This means that we look for what gets the best results and make that standard practice with all teams, working now and for future projects.

Additionally, the customer focus demands this. If we have two teams doing projects for the same customer, the customer expects a consistent experience with us in both. It is rather like how McDonalds fries taste the same in Texas as they do in Tokyo. McDonalds has standardized their french fry making process, so the customer experience is the same.

This rationale can help initially resistant staff agree to adapt and move forward.

[]14.32. Test Process Changes with Experiments Before Going Bigger

Long ago, a speaker in Dallas said that if you are trying to find a success that has not been discovered yet, make small investments more often rather than one huge investment that may miss.

He likened it to shooting arrows at a target that sits down range in a fog. You can not see the target. You can shoot an arrow and possibly hear a “thunk” noise as the arrow hits the target or no sound as the arrow misses the hidden target. His point was that instead a single large golden or carbon fiber expensive arrow, with great hopes shot into the fog, make many smaller cheaper arrows of bamboo or wood and shoot in a consistent pattern to try to find the target. “Once you hear thunk,” he said, “then you can shoot all your remaining arrows at the location that succeeded.”

When trying process changes, make experiments in small sizes and see what the data shows before expanding. This may lead to smaller failures early on, with a larger return once you hit the target and can repeatedly shoot at the target. This small-test approach also allows you more experiment cycles and team learning within the available budget.

[]14.33. Have Someone Focused on the Customer

When someone is focused on your customer, you may notice changes in their environment that impact the training development project. You may find you didn’t communicate enough about Lean-Agile to someone in their organization. You may find out about people in the customer’s organization that are detracting from the project.

Staying focused on the customer is also how you hear good news about how you’re doing. Write down the good stuff and share it back with the customer and your own leadership. Use it as testimonials for your next business pursuit.

[]14.34. Great Staff is the Root Cause of Success

Great staff is the root cause of success even in Lean-Agile. Be sure your selection process is rigorous and your on-boarding process is sufficient to find and get such staff.

Another approach is to grow your own great staff by making the process clear, providing opportunities for staff with less experience to gain capability and confidence as they work.

Express appreciation as if the employees are volunteers, because people appreciate it when you notice their efforts.

[]14.35. Our Dissenter Helped Us Improve

A dissenter during a change initiative can challenge the thinking that goes into the change. We had a dissenter that questioned all the changes that Lean-Agile brings. Communicating changes, well enough to satisfy that person helped the overall change to Lean-Agile include better rationale. When we later brought in additional staff, this codified rationale was helpful in training new people.

You have to judge how much dissent you’ll allow in a change initiative. Too much can cause an effective mutiny. None can leave you with incomplete implementations. Having a certain degree of dissent can be a good thing.

[]14.36. Welcome Novel Ideas and Solutions from Team Members

In a dramatic change like moving to Lean-Agile, you really need the ideas of the entire development team to make the switch work.

Just like listening to everyone’s ideas while looking at the Kanban board helps teams improve as they build courseware, listening to new ideas and solution proposals from your change team can help make this change initiative stick better.

Often the person with the idea also has the energy and desire to pursue an experiment and bring back the results data.

[]14.37. Tiered Kanban Boards Help Show the Big Picture

Tiered Kanban boards helped avoid micromanaging and allowed a better connection to what is happening.

If you’re in a small organization, the board can be the entire update process.

In bigger companies that are used to lots of separate reporting mechanisms, you may have to reformat the Kanban information to lists, power point slides and other specially formatted reports.

Pulling team leads into a daily synchronization meeting at the next higher tier can help see the progress for the entire effort.

We do this with a separate board for tracking all projects, which is different from the specific project’s Kanban board that only shows work items for that project.

You may need to get a waiver from converting your visual data into multiple reporting formats, what Lean calls waste.

[]14.38. Teams Can Change the Process

To stay Lean-Agile, we let teams can change the process from the default process steps after a context change. The team can make the change with fewer controls, remaining agile in their responses.

We have a standard process that the team can tailor if needed. We allow their initiative to make the change, and we want reports that it has been done and what the impact of the change has been to date.

[]14.39. Iteration Reviews are “No Homework” Activities for Buyer/Customer Stakeholders

One customer asked that we provide the courseware increment sooner than the review date so that the stakeholders would have time to review it before providing their feedback.

We explained that we have no expectation that the feedback during the review is a homework assignment, and that we really wanted to come as they are and tell us what comes to mind as they inspect our courseware increment. If they need a formal review for the customer’s own processes, then we added a formal review after the iteration review/demo and let the customer stakeholders submit formal change requests (CRs) against that increment within a specific timeframe.

For one customer, it was three days. For another, they needed nine days. So we then needed to explain the impact of taking nine of the ten days of the next iteration to document CRs from the prior iteration would mean that we could not make those changes until the iteration after next. They understood, but needed that time for their stakeholders to review. So, we did it different ways for different customers.

[]15. Roles in Agile

Roles in Kanban are unchanged from your current roles. It is only when you apply Scrum that formal roles need to change.

[]15.1. Role of the Agile Coach

The Agile Coach helps keep the team focused on delivering work item cards and fulfilling the iteration goals and commitments they have made to the customer.

If your organization uses Team Leads, the Agile Coach assists the Lead in the following ways:

  1. {color:#000;}Ensures visual management and transparency.

  1. {color:#000;}Helps with any obstacles between the product owner and the development team.
  1. {color:#000;}Coaches the product owner on Agile methods.
  1. {color:#000;}Helps the team decompose work items into similarly sized chunks
  1. {color:#000;}Facilitates creativity and empowerment in the development team.
  1. {color:#000;}Coaches the development team to use retrospectives to improve their own team productivity.
  1. {color:#000;}Coaches the development team to use retrospectives to improve their tool chains and practices and tools.
  1. {color:#000;}Coaches the development team to ensure each product increment is potentially deliverable, pending reviewer comments.
  1. {color:#000;}Ensures the team updates the Kanban cards to make the team’s current progress status and total WIP visible to the team and all stakeholders with access to the physical or virtual Kanban board.
  1. {color:#000;}Shows the Team Lead how to modify the Kanban board as the process needs to change.
  1. {color:#000;}Implements Agile.
  1. {color:#000;}Teaches and coaches people on how to perform in the various roles.
  1. {color:#000;}Helps the team to lead changes in the larger organization.
  1. {color:#000;}Monitors the team as they remove impediments. Makes suggestions as appropriate to help the team become self sufficient using Agile.
  1. {color:#000;}Guides the team in the use of Agile.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. We don’t use the term Scrum Master, but Agile Coach instead because fewer training people are familiar with Scrum so far.|

[]15.2. Role of the Agile Team Member

[]15.2.1. Traditional Role

  1. {color:#000;}Do work assigned (push vs. pull)
  1. {color:#000;}Help the team coordinate and collaborate (weekly, monthly)

[]15.2.2. Agile Modifications to the Role

  1. {color:#000;}Individuals select (pull) their own work items from the prioritized buffer column (ready, backlog or “done” from prior step).
  1. {color:#000;}Although an individual fulfills one or more roles on a project team, that does not imply that the person is constrained to doing only specific types of work.
  1. {color:#000;}Members coordinate and synchronize with the team daily at the daily standup meeting and impromptu discussions between individuals.
  1. {color:#000;}The team member helps keep the iteration cadence. All team members are required to attend.
  1. {color:#000;}Team members move their work item cards BEFORE the daily standup. Team members don’t spend time verbally providing status, instead the Kanban board visually provides the status to all so the team can collectively address traffic jams, bottlenecks and constraints on the board.
  1. {color:#000;}Team members are prepared to pull cards and commit to what they will do today.
  1. {color:#000;}Team members tell the team what is impeding their work.
  1. {color:#000;}Team members make working courseware functionality available as early as possible.
  1. {color:#000;}Team members test their own work items before calling it “done” to ensure it works as expected. The goal is to find and fix defects. Test to the risk. The riskier something is, the more it needs to be reviewed and tested.
  1. {color:#000;}Team members continuously integrate their components into the courseware build using the courseware authoring software.
  1. {color:#000;}Everyone commits to the work. The team commits to accomplishing the iteration goal by the end of the two-week iteration. Individuals commit to the work items they pull and add their avatar to the work item card to signal ownership to others.
  1. {color:#000;}Everyone contributes.
  1. {color:#000;}No one stays only to their specialty.

[]15.3. Role of the Agile Quality Assurance Content Inspector/Functionality Tester

Anyone but the person who developed the component being tested can be a Content Inspector/Functionality Tester. This is not a hired position, but a role all existing team members take on.

Content Inspectors/Functionality Testers do the following:

  1. {color:#000;}Pull cards into the test column/process step on the Kanban board and test them.

  1. {color:#000;}Perform training product verification and validation testing using manual and automated test tools.
  1. {color:#000;}Update their own work item status cards daily.
  1. {color:#000;}Document defects on Defect Kanban Cards.
  1. {color:#000;}Test against planned test cases listed in the Test Checklist for the appropriate type of item.
  1. {color:#000;}Validate requirements compliance (for example, accessibility, SCORM, xAPI)
  1. {color:#000;}Test for learner experience.
  1. {color:#000;}Apply test case checklists.
  1. {color:#000;}Effectively communicate with the team and management.
  1. {color:#000;}Playtest through games or simulations or courseware.
  1. {color:#000;}Recommend new or revised test cases to be included in the test case checklists.

[]15.4. Role of the Product Owner

In the Agile Scrum practices there should be a product owner.

Product Owners do the following:

  1. {color:#000;}Has the responsibility for the backlog and ensuring it is prioritized.

  1. {color:#000;}Represents the customer to the development team.
  1. {color:#000;}Must be available to the team at any time, but especially during iteration planning meetings and iteration review meetings.
  1. {color:#000;}Act as the stakeholder’s representative.
  1. {color:#000;}Prioritizes the courseware product backlog.
  1. {color:#000;}Aggregate inputs from customers, stakeholders and other interested parties to form a single view of the prioritized backlog.
  1. {color:#000;}Accept or not accept work results of the development team.
  1. {color:#000;}Update priorities at the end of every iteration cycle.
  1. {color:#000;}Balance the interests of competing stakeholders.

[]15.5. Role of the Team Lead

For organizations not ready yet for 100% self managing teams, there is often a Team Lead or some similar job title related to supervision.

For learning experience development, it is important for this role to have an in-depth understanding of learning experience development. This may include experience in web-based training, blended instructor-led training or training simulation.

[]15.5.1. Traditional Team Lead Role (Non-Agile)

Traditional Team Leads guide the media specialists and software developers to ensure their components are built to address the learning objectives as intended.

Traditional Team Leads report progress to management.

[]15.5.2. Agile Modifications to Team Lead Role

In Agile, Team Lead roles typically perform the following:

  1. {color:#000;}Modifies the schedule to depict Agile iteration/courseware increments every two weeks.

  1. {color:#000;}Invites management to view the Kanban board at any time to get information they need on progress.

|={color:#000;background:rgba(0, 0, 0, 0);}. Caution|<{background:rgba(0, 0, 0, 0);}.
p<{color:#000;}. Much of the Agile literature seems to indicate that the ideal is that leaders simply view the Kanban board that radiates the information leadership needs to assess progress.

However, in many organizations where the particular courseware project is one of many in the manager’s portfolio, the manager cannot or will not take the time to use the Kanban as their only assessment input. Overseeing large portfolios of projects or programs often means the manager may not have the time to personally prepare all the reporting.

Often, managers are not located near the development team. Using a self-serve Kanban board requires the manager to go to the Kanban board, either walking to a physical board or signing on to a virtual Kanban board app. Some managers still have to forward progress updates to their next higher level of management in a format that leader expects.

Using a Kanban board also delegates up the reporting effort back to the manager, instead of the typical non-Agile situation where the reporting effort is delegated down to the Team Lead.

Managers in these situations will typically continue to ask for periodic reports in the normal format that they use to monitor the health of their entire portfolio of projects.

In this case, the Kanban still helps because it provides the Team Lead with the information needed to provide that report to management without interrupting the team members every time this management update report is due.

You may need to change the management culture if you desire to use Tiered Kanbans as the primary reporting mechanism. Simply transferring the reporting effort (Lean transaction waste?) from the Team Lead to the manager is a hard sell in many organizations. |

[]15.6. Role of the Manager

The manager’s role tends to vary by organization, so take this list with a grain of salt and adapt it to your situation.

[]15.6.1. Traditional Project Manager (Non-Agile)

This role is often the individual accountable for the effective management (cost, schedule, quality) of a project. This role is accountable for the overall management and successful completion of the project.

  1. {color:#000;}Establishes the project budget

  1. {color:#000;}Ensures project communication
  1. {color:#000;}Ensures project definition (objectives, work breakdown structure, identify resource requirements)
  1. {color:#000;}Ensures project planning

–Assigns responsibility

–Sequences deliverables

–Schedules deliverables

–Schedules resources

–Develops a risk and opportunity plan to assess and plan prevention or capitalizing actions

–Establishes the earned value system if used

  1. {color:#000;}Ensures project implementation

–Assembles the team

–Monitors project

–Modifies project

–Closes out and evaluates the project

  1. {color:#000;}Gets status from the development team regularly

  1. {color:#000;}Arranges customer meetings as required
  1. {color:#000;}Approves purchasing and subcontracting (if required)
  1. {color:#000;}Partners with customers to ensure project plan and schedule is doable (cost, schedule, technical)
  1. {color:#000;}Sets performance standards for the project participants

[]15.6.2. Agile Modifications to the Manager Role

  1. {color:#000;}Some of the traditional role is still included.
  1. {color:#000;}Manager helps manage the customer’s expectations about Lean-Agile.
  1. {color:#000;}Ensures visual management—a Kanban board to radiate data for reporting.

[]15.7. Role Adaptations for Small Shops

In companies where training is not the main business, their training organizations are typically just a handful of people. So often in smaller companies, people have to wear multiple role hats.

[]15.8. A Day in the Life

We have to work within a traditional project management culture for the enterprise, so we still perform planning up-front for the project-level/program-level.

Analysis moving analysis to Agile could be the scope of an entire other book, so we will not address it in this day in the life example.

Design, meaning the course design document—course level design. Moving design to Agile could be the scope of an entire other book, so we will not address it in this day in the life example. We have written a section of this book just on Agile Design.

Learning experience development is where the bulk of our day in the life will be illustrated.

In Lean-Agile, the instructional/LX designer gathers the dispersed team on the phone at the agreed time, and someone starts a 15 minute timer. The team looks at the Kanban board together (virtual or physical), and they discuss what needs to be done, basically applying PDCA at the daily level to keep the project on track. A team member added a “blocker” visual signal to the work item card that cannot be done now because SME#1 is unavailable. The entire team sees the blocker signal and there is a brief discussion about it. Everyone can see if everyone else’s committed work is progressing as they said it would or if there are problems. There is no sliding along in this visual environment. All of them are involved in solving the team’s problems together. Observers have been asked by the project manager not to interrupt during that 15 min daily synchronization meeting, but to hold their questions or comments until later. The Agile Coach may remind the team to apply a particular Agile principle slightly differently as needed.

After the meeting, the instructional/LX designer may have a one-on-one discussion with one of the team members to clarify details on something or another. Other team members will look at the board and pull a work item card into their WIP column and begin work. They all heard the customer say they wanted XYZ different in the next iteration, so they all do that as they do their work items. When completing a work item, the team member “tests” the work item against the checklist that we use of manual test cases to confirm that the quality is sufficient to call the work items “done.” If the work item passes the test cases in the checklist, then the team member pulls the work item card from the Work In Process column on the Kanban board and into the “Done” column, which is the ready column for the next step in the development workflow/process.

Because this is training after all, the instructional/LX designer’s storyboard progress is the drum beat of the team. Without that, the team stalls. So assuming the instructional/LX designer is caught up on the storyboard, then the other team members take work items to make the storyboard a reality. If the instructional/LX designer has to do more storyboards, then the team pulls the defect/change request work item cards and fixes those. Everyone knows they all have to demo/show the customer stakeholders another increment of working courseware in the two-week countdown, so a sense of urgency is present. They want to get it right, so the customer stakeholders like what they see.

The Agile Coach sees that new team member, JoAnn, does not seem to understand how to apply the current Agile methods and takes a minute after the meeting to coach her.

The team may indicate they have an impediment, in that SME#1 is unavailable, and the lead SME is also not responding. The project manager may have to help fix the SME issue to remove the impediment.

The project manager needs an update on where the team’s progress is, but instead of interrupting their work, he or she simply looks at the Kanban board to see where the progress is and if there is a crisis that need to be addressed. The project manager also buffers the team from other interruptions, such as organizational metrics seekers by providing that data for them, so the team can work on the learning objectives without context switching waste.

The project manager may have to remind an organizational leader about WIP limits because they asked why so few work items are showing “any progress” at a time.

The SME’s manager calls to bear the bad news that the loss of the SME#1 is unavoidable, so the Product Owner pushes that LO work item to the next iteration and brings another LO from that next iteration forward in its place. The instructional/LX designer notes that he or she will need to point out at the start of the next iteration review/demo that LO#34 moved to the next iteration and that LO#47 moved up to this iteration.

The instructional/LX designer calls SME#4 to remind her that the job-relevant case study details she is working on are due tomorrow.

At the end of the day, one or more LOs are done—meaning ready for student use—and put into the “Ready for Iteration Review” ready queue. This is the one piece flow in action.

This day in the life does not include the pilot because that is the same as without Agile so far. I’m open to your ideas about how to improve that. We do track the defect/change request work item cards on the Kanban board after the pilot to see how close we are to delivery of the post-pilot courseware.

[]16. Next Steps[]

[]16.1. Separate Teams for Player Development and Content Development Teams

If your courseware delivery includes a custom or configurable content player application (for example, rendering HTML, XML or JSON) that you develop and deliver to your customer, then consider having separate development teams for the content player and the content itself.

This can help the boards have a manageable number of cards in process at the same time. These groups also have separate concerns. Players are more software development and functionality-centric while courseware content is more learner-centric. The process steps for software development are different from the process steps for instructional courseware content.

[]16.2. Shape Customer Expectations Before Contract Execution

Attempt to shape customer expectations before contract execution with the ability to trade off functionality/content scope for the highest priorities.

Ideally, negotiate Lean-Agile contract provisions that allow swapping of equal amounts of work effort for newer, higher priority work effort to allow the customer to change as they discover more and to allow the development team to be Agile and to adjust.

|={color:#000;background:rgba(0, 0, 0, 0);}. Caution|<{color:#000;background:rgba(0, 0, 0, 0);}. Be aware that fixed scope (requirements) is closer to the Waterfall end of the project management continuum, making all requirements the same priority. You have to help the potential customer resolve their concerns about changing to an unfamiliar project management approach for training products.|

[]16.3. Senior Leadership Has Got Your Back

We are grateful for our leadership team who were willing to experiment with Lean-Agile in business critical projects and programs.

Learn as much as you can on internal projects before moving to customer projects/programs.

To help you achieve this, work to develop the credibility to engender patience with senior leaders so when your situation moves into the complex context and you move to more of an experimental mode than a predictable mode, they will not try to over control the situation when the results are not what you aimed for, preempting the chance to see the patterns emerge.

You need support from senior leaders to find success before the organization gives up on Lean-Agile.

[]16.4. Getting Started–Make Your Own Way

So you want to apply Lean-Agile in your circumstances. How do you get started? Use this list to get your thinking cap warmed up. Have fun.

  1. {color:#000;}What is the problem you are trying to improve by applying Lean-Agile? How urgent is the problem? Articulate why you are introducing this new way so people can understand why they should change.

  1. {color:#000;}Where are you now? Measure existing productivity before introducing Lean-Agile so you can compare and see progress.
  1. {color:#000;}Lead yourself first. If you want to lead others down the Lean-Agile path, you need to know it well enough to help them come with you. Learn the Agile principles. Apply them on a personal project to learn and make mistakes in private. What can you do within your own stewardship first in order to learn? Seek under-the-radar experiments that are low harm but good chances to try Lean-Agile on a small scale. Identify gaps in your knowledge and skills, grow and try another small-scale experiment.
  1. {color:#000;}Enable consistent performance. Make and use checklists as performance aids. Pilots and medical doctors do this even with all of their training.
  1. {color:#000;}Articulate why you think Lean-Agile can help your situation. Identify your reasons for change.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. If you have an existing organizational crisis wave you can surf, do so to help nudge people to see that the change is important and urgent. If not, consider how you or the leadership sponsor can create a crisis or a sense of urgency for the change to Lean-Agile. Increase urgency to nudge people. Inspire people to move and make objectives real and relevant.|

  1. {color:#000;}Figure out who cares, who are your stakeholders, in your situation and decide how to manage their expectations.
  1. {color:#000;}Know who the organizational antibodies are that, like white blood cells, will try to snuff out Lean-Agile as if it is an invading virus to the organizational body. Get the sponsor to give you a waiver to current processes for the duration of the experiment with Lean-Agile, similar to Kublai Khan’s golden tablet, like a letter of safe passage, worked for Marco Polo as he traveled across Asia.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Organizations are complex adaptive systems (CAS), so apply learning about CAS to help your effort succeed.|

  1. {color:#000;}Decide whether or not you need an organizational sponsor(s) from the organization’s senior leadership. Plan the involvement and project activities of the change sponsor(s).
  1. {color:#000;}Facilitate. Determine what you will need to do to get buy-in for the changes from those involved and affected, directly or indirectly.
  1. {color:#000;}Lead. Burn the ships like early explorers landing in the Americas to remove the option to retreat to the status quo. People often don’t want to change, and they often won’t, unless they have good reason to. You and the leadership sponsor must commit to the change. Talk consistently with the change. Let people know it is required. Behave consistently with the change.
  1. {color:#000;}Deal with resistant and complaining Eeyore characters (see Winnie-the-Pooh for examples). Some people will resist any change and will stand on the sidelines and point out loudly to all who will listen any flaw in your change effort. A persistent resistance Eeyore can threaten a change project. Have a plan to deal with that before it happens.
  1. {color:#000;}Determine who needs to be involved in the means and ends.
  1. {color:#000;}Plan your communications. How will you tell everyone who’s affected about the changes? How will you share evidence to help them empirically support you?
  1. {color:#000;}Decide how big of a bite or scope, you will attempt on the first iteration and the second iteration.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Consider starting small and succeeding with a quick win before expanding. Success begets success. It helps with leadership and with getting more people to buy in. Consider your own WIP limits to make this successful.|

  1. {color:#000;}Get a formal mandate for the project if that fits your organization process; otherwise, ignore this.
  1. {color:#000;}Adjust Lean-Agile to your context. Take the principles and apply them for your unique situation and see improvements in your learning experience development results.
  1. {color:#000;}State your why and project objectives, estimate costs involved and list any requirements, constraints and assumptions.
  1. {color:#000;}Overlay Kanban over whatever process you do now. It is a light-weight Agile method that does not require much change.
  1. {color:#000;}See the task list in Appendix A for skills that need to be learned and applied .
  1. {color:#000;} List what you will deliver and develop your work breakdown structure (WBS) to get it done - whether your WBS is hardware-, product-, service- or process-oriented.
  1. {color:#000;}Identify resources, whose help is needed, supplies, software license costs and constraints.
  1. {color:#000;}Ensure participant readiness. Decide how you will get people ready to adapt to Lean-Agile changes. Ensure they have the right information, training and help. Don’t breed snipers. An ounce of prevention is better than a pound of pain.
  1. {color:#000;}Who are your go-to, core team of change makers with the right attitudes and skills? Meet with them one-on-one to invite their contribution and gain commitment. Get them on board first. Let them know you will remove obstacles, listen to constructive feedback, support their efforts and that you will reward and recognize their progress and achievements.
  1. {color:#000;}Set the conditions for success.
  1. {color:#000;}Train your core team well, so they’re not stranded while you’re busy.
  1. {color:#000;}Set WIP limits for your core team so they don’t get slowed down, potentially threatening the entire change initiative.
  1. {color:#000;}Assign a Lean-Agile coach or two and ensure they are competent in both coaching and in the technical considerations of Lean-Agile implementations.
  1. {color:#000;}Assign responsibility, identify who is responsible for each part of the plan and commit them.
  1. {color:#000;}Plan iterations to sequence time-phase any deliverables.
  1. {color:#000;}Schedule resources.
  1. {color:#000;}Identify a team you can lead into Lean-Agile. Stack the initial team if possible.
  1. {color:#000;}Conduct high-level planning.
  1. {color:#000;}Identify risks and opportunities by probability and magnitude of impact. Decide what to protect against by the impact. Identify causes and ways to mitigate problem causes or encourage opportunity causes. Decide what triggers contingent actions.
  1. {color:#000;}Begin implementation, use a Kanban for the change implementation team to practice what you preach or “eat your own dog food” as software people sometimes say.
  1. {color:#000;}Iterate.
  1. {color:#000;}Repeat the main messages often, like advertising. Telling people once and stopping is less effective during change efforts.
  1. {color:#000;}Monitor the project visually against objectives, budget and schedule (if required). Try to avoid a Gantt chart if possible.
  1. {color:#000;}Experiment. Apply PDCA frequently. Consider daily over weekly.
  1. {color:#000;}Evaluate. Assess and address how the changes will affect people. How close did you get to your planned objectives?
  1. {color:#000;}Celebrate and recognize any success.
  1. {color:#000;}Recommend organizational design proposals or job-redesign proposals as appropriate to restructure the organization to better use Lean-Agile.
  1. {color:#000;}Share any successes within your organization appropriately.
  1. {color:#000;}Decide whether you should or can merge the Lean-Agile approach with Capability Maturity Model Integration (CMMI) if your organization has to use CMMI.
  1. {color:#000;}Work towards convincing a larger organization to apply as much of Lean-Agile as you can persuade them to use.
  1. {color:#000;}Then share what worked and didn’t work for you at industry conferences or write your book so we can read it.

Thank you for reading our book about applying Lean-Agile to your work. Good luck.

[]17. Contact Information[]

[]17.1. Contact Information if Assistance is Desired

Contact Raytheon Intelligence, Information and Services (IIS), Engineering and Technology, Training and Logistics Services if you need assistance.

We are located at:

22265 Pacific Boulevard

Dulles, Virginia 20166 USA

(972) 272-0515



[]17.2. Collaborators for this Book

Collaborators for this book included Kevin Lanham.

Reviewers included Terry Stroud, Erica Bourne, Rich Edwards, Mark Sharbak, Amanda Cognet and Peter DeRosa.

Editors included Dawn Bona.

Cover art by M. David Figueroa.

[]17.3. We Used a Kanban Board to Write this Book

We used an Agile Kanban board with sticky notes to track the work items of preparing this book. It helped us stay on track since the work was stretched out around other projects, due to poor WIP limits at the time. The Kanban helped us get back on task quickly.

We used a product backlog column for the whole book, and iteration backlog column for the times we could do focused work on it, a work in process column and a done column.

For learning experience development teams, our boards are more detailed with a column for each process step, but for this book we stayed simple. Mostly to help communicate about Agile, and also because our specific steps are proprietary. We just needed a way to track the work remaining. We annotated time spent on each sticky note, so we could estimate the total authoring time for the first draft, editing, et cetera. Capturing performance metrics helps estimate future projects that are similar.

With courseware projects, we have the challenge that the teams are virtual with people geographically dispersed across the United States. In that context, we use virtual Kanban boards in software applications rather than a physical white board because we cannot colocate all the team members. We have a similar constraint in that some management is also not at a single site, so they can’t just wander into our work area, or what Lean calls gemba, and see our Kanban board. So we have extra licenses for them to see the virtual board.

[]18. Colophon

This content was originally conceived and drafted in eXtensible Markup Language (XML)(http://www.w3.org/XML ) using a commercial XML editor and occasionally Notepad++ ( https://notepad-plus-plus.org). The original XML schema used was the Darwin Information Typing Architecture (DITA) bookmap DTD/schema that IBM open sourced as the DITA Open Tool Kit (DITA OT) (http://www.dita-ot.org).

About halfway through the project, we changed the tools pipeline to use plain text for authoring and to use DocBook XML as simply a transient format, making version control easier with free and open source tool git (https://git-scm.com). Git worked very well as version control for the asciidoc source files. With the help of a 184-lines-of-code XSLT 2.0 (http://www.w3.org/TR/xslt20) file and Saxon (http://www.saxonica.com), we converted the DITA XML to AsciiDoc (http://asciidoc.org) markup text in seconds with follow-on minor clean up by hand. Then, we were back to writing again with the new tool chain.

We then continued the process of authoring, editing and revising the content using the text-based markup language called AsciiDoc using various free and open source text editors such as Notepad++ and mostly Atom ( https://atom.io/). The markup was built in modular pieces following the model of the DITA XML methods with the main AsciiDoc file acting as a module map to the component files using the AsciiDoctor include statement. Although the topic-based modularity was initially due to original information architecture from in DITA XML, it proved helpful, and we maintained the modular architecture of the content throughout the project. For editing review, we added automated HTML5 formatting back to the AsciiDoc text source using a free and open source tool called AsciiDoctor (http://asciidoctor.org) that runs on the Ruby command line (http://www.ruby-lang.org). The HTML file could also be opened by Microsoft Word (https://products.office.com/en-us/word) for reviewers not yet comfortable with text processing. We also used Pandoc (http://pandoc.org/) for conversion of Asciidoctor HTML to MS Word files. Reviewers made change annotations in MS Word that were incorporated into the AsciiDoc source with versioning using git.

For other reviewers and for publishing, the plain text AsciiDoc markup was converted to PDF by transforming the AsciiDoc markup to DocBook XML using AsciiDoctor. The DocBook XML was transformed to XSL-Formatting Objects (XSL-FO) using Saxon and DocBook XSL stylesheets. The XSL-FO was converted to PDF using FOP (https://xmlgraphics.apache.org/fop), an open source XSL-FO engine. The conversion from asciidoc source to PDF was clean and worked well.

The ePub (http://idpf.org/epub) release version was originally planned to be made with AsciiDoctor-epub3, but that did not work out as smoothly as planned. The app was in alpha, so we moved on rather than troubleshoot the issues. Our intended distribution site (http://Shakespir.com) wanted MS Word as the input. So we used asciidoctor to convert the asciidoc source files to HTML5, then we used pandoc to convert the HTML to MS Word in a few minutes. That choice moved all the footnotes to the end of the book rather in the chapters like it did for the PDF. Then we found out Shakespir does not convert tables at all. So, after experimenting with ePub creation using pandoc and Caliber, we happened upon the fact that Apple Pages exports really well to ePub format, so we loaded the MS Word file into Pages and exported and it worked. Moving to ePub we also removed the PDF index. We adjusted image sizes from PDF size appropriate to eReader size appropriate. We used Calibre to add the book cover and edit the eBook metadata. With more time and cost we could have continued the journey for the perfect ePub eBook, but this path was sufficient for our needs to offer this book to the L&D community without price.

Use AsciiDoc for document markup. Really. It’s actually readable by humans, easier to parse and way more flexible than XML.[77]

— Linus Torvalds

At first glance this tool chain may sound complicated; however, like Linus Torvalds, we found working with AsciiDoc far easier than working directly with HTML5 elements, DITA XML, DocBook XML or S1000D XML. This is after having used all of these tagging schemas professionally. So when you have technical training or technical writing projects, we recommend you start directly in AsciiDoc and automatically convert to XML as needed rather than starting in XML. The text files with AsciiDoc markup are easy for human reviewers to inspect and edit. The media get inserted by reference like with HTML and XML. AsciiDoc makes for quicker training storyboards too. The structure offered by the AsciiDoc markup allows easy transformation between formats. AsciiDoc provides semantic markup with abbreviated markup rather than the more verbose HTML and XML elements and attributes. Of course, all such markup scheme implementations may initially be helped by having markup job aids.

[]Appendix A: Lean-Agile Job Tasks for the Buyers[]

[]A.1. Job Tasks for Lean-Agile Courseware Buyers

This section is aimed at training buyers.

To better articulate our discoveries about what was really needed on the job for people transitioning to buying courseware using Lean-Agile, we performed a task analysis and developed the following Task List of things people need to be able to do using Lean-Agile on courseware product development projects.

You may need some of these, and others that are not listed. Use this as a start point and adapt for your unique circumstances.

[]A.1.1. Review Current Strategy

  1. {color:#000;}Define stakeholder and customer needs

[]A.1.2. New Strategy–Decide Whether to Use Lean-Agile

  1. {color:#000;}Identify timescales and resources available.
  1. {color:#000;}Authorize funding the project as you have always done.
  1. {color:#000;}Determine to take advantage of improved cost and schedule performance for projects requiring contractors/suppliers use Lean-Agile.
  1. {color:#000;}Gain senior leadership buy-in and support up front for using Lean-Agile for cost, performance, and risk mitigation goals. Their support may be necessary to overcome turf tussles, keep stakeholders onboard during the project and to remind all that learning what works is valuable for future efforts. Status quo will not get the gains that Lean-Agile can provide. Without leadership, status quo will win out.
  1. {color:#000;}In cooperation with legal counsel, determine the Agile contract terms and conditions you will use to support your goals.
  1. {color:#000;}Gain consensus from all approvers and stakeholders that this Lean-Agile project will freeze cost and time, and allow changing scope (content and functionality) based on highest buyer value. Therefore scope juggling may occur during iteration planning meetings.
  1. {color:#000;}Determine if the buyer’s staff needs to be trained on how to work within the Lean-Agile approach.

Be sure any staff training has already tailored for training services and products rather than for software or manufacturing. The principles indicate different methods are appropriate for training.

  1. {color:#000;}Develop your business case for Lean-Agile.

[]A.1.3. New Strategy–Determine How Much Lean-Agile your Organization can Support Now

  1. {color:#000;}Determine how much Lean-Agile your organization can support now with your current culture.
  1. {color:#000;}Explicitly tell the contractor/supplier your chosen degree of Lean-Agile for this project. We recommend a face-to-face meeting for this communication.
<{background:rgba(0, 0, 0, 0);}.
p((((((((((={color:#000;}. * Caution*
<{background:rgba(0, 0, 0, 0);}.
p((((((<{color:#000;}. [* To avoid buyer and contractor/supplier expectation mismatches and much consequent Lean waste, the buyer should share their Lean-Agile hybrid tailoring decisions. A contractor/supplier assumption that the buyer has decided to go 100% Lean-Agile when they have not will create communications waste and relationship issues. *]
  1. {color:#000;}Buyer’s leadership intervenes if the buyer organization begins to revert back into traditional waterfall approach mid-project and does not stick to the Lean-Agile hybrid tailoring decisions you have made as an organization.
  1. {color:#000;}Decide whether courseware instructional/LX design can emerge in the iterations or if a courseware design document is desired up front.
  1. {color:#000;}Review buyer’s existing governance policies to ensure they allow for Lean-Agile and don’t create unintended obstacles.
  1. {color:#000;}Create new governance policies that accommodate Lean-Agile as appropriate.
  1. {color:#000;}Ensure you can allocate SMEs and stakeholders for iteration demos/reviews.
  1. {color:#000;}Develop a new culture plan to realign your culture to more successfully fit Lean-Agile.

[]A.1.4. Market Research

  1. {color:#000;}Identify contractors/suppliers.
  1. {color:#000;}Interview contractors/suppliers as needed.
  1. {color:#000;}Notice training conference presentations which are beginning to include using Agile for courseware projects.
  1. {color:#000;}Interview other organizations that have attempted Lean-Agile to see what they learned and to avoid mistakes they unintentionally made.
  1. {color:#000;}Look at the types of contracts being used to purchase Agile software. Determine which contract terms and conditions cross domains into training and which do not.

|={color:#000;background:rgba(0, 0, 0, 0);}. Tip|<{color:#000;background:rgba(0, 0, 0, 0);}. See how your organization buys custom software products. It is likely that those involved already use Agile and that you may be able to leverage from their experience.|

[]A.1.5. Define the Project

|={color:#000;background:rgba(0, 0, 0, 0);}. Caution|<{color:#000;background:rgba(0, 0, 0, 0);}. For traditional projects, requirements scope is fixed while costs and schedule are variable. For Lean-Agile projects, costs and schedule are fixed, and requirements scope is variable. This is key to the benefits that Lean-Agile can provide. Efforts to make all three elements fixed will increase project risk.|

  1. {color:#000;}Treat the budget and schedule for the project as fixed.
  1. {color:#000;}Treat the requirements scope as variable and prioritized for highest buyer value to get the highest priority work at the end of the cost and schedule. Lower priority requirements may not make it. This is okay and is the goal with Lean-Agile.
  1. {color:#000;}Determine the training need and gaps as you have always done (i.e. Training Needs Analysis).
  1. {color:#000;}State the performance deficiencies that will be addressed.
  1. {color:#000;}Document goals and objectives. If you are a government buyer, create the performance work statement (PWS) or statement of objectives (SOO) in a way that does not unintentionally constrain Lean-Agile.
  1. {color:#000;}Establish design assumptions, freedoms and constraints (i.e. exploiting existing training, accessibility, technology infrastructure, etc.)
  1. {color:#000;}Warn stakeholders that there will be no long up front detailed plan, but that more like rolling wave, details will be planned closer to the work beginning.
  1. {color:#000;}Draft the requirements
  1. {color:#000;}Prioritize the requirements in highest-to-lowest customer/stakeholder value order—to help trade-off lower priority requirements for the highest priority requirements.
  1. {color:#000;}Prioritize the most technically challenging and highest priority work items the highest as a risk mitigation method. This is instead of the order in which the learner experiences it.
  1. {color:#000;}Visualize the requirements as a prioritized stack of backlog items or Kanban cards.
  1. {color:#000;}Require contractors/suppliers to provide visibility into complex technology components that are in process for your stakeholders. The preferred way is seeing snapshots of their Kanban board.
  1. {color:#000;}Set the iteration length (for example two weeks, or four weeks) based on the degree of risk mitigation desired. Two weeks has less risk than four weeks.
  1. {color:#000;}Read requirements written in user story format—“As a role, I want functionality/content, so that business value.”
  1. {color:#000;}If defining interaction types, then describe your desired interaction types as user stories.
  1. {color:#000;}Require a prototype of each interaction type in early iterations to mitigate risk.
  1. {color:#000;}Identify acceptance criteria for requirements.
  1. {color:#000;}Build your own risk register or require the contractor/supplier to build it.
  1. {color:#000;}If you are a government buyer, be sure that your Independent Government Estimate (IGE) uses a similar approach or is factored appropriately for the Lean-Agile project management approach.
  1. {color:#000;}If you are a government buyer, be sure that your Quality Assurance Surveillance Plan (QASP) methods of assessment account for the built-in quality approach in Lean-Agile rather than forcing additional steps all at the end.
<{background:rgba(0, 0, 0, 0);}.
p((((((((((={color:#000;}. * Note*
<{background:rgba(0, 0, 0, 0);}.
p((((((<{color:#000;}. _ Look to how the QASP is created for Agile software projects for a good start. Keep in mind that learning experience development has less automated testing possible today than software, so test driven development is still a far off objective today._
  1. {color:#000;}Evaluate your QASP to see if it unintentionally creates Lean waste.
  1. {color:#000;}Establish stakeholder consensus.

[]A.1.6. Clarify Roles and Responsibilities from All Involved

Let stakeholders know what to expect from a Lean-Agile courseware project.

  1. {color:#000;}At the beginning, identify someone to document your lessons learned in the experiment to adopt Lean-Agile to facilitate an easier path on future experiments.

  1. {color:#000;}Update your process overview for stakeholders not familiar with details of training design and development.
  1. {color:#000;}Set the expectation for no Gantt charts. Fixed iterations every two weeks are the schedule. The plan does not drive the Lean-Agile project, but the value (priority) does.
  1. {color:#000;}Clarify with your auditing group what changes will be necessary to use Lean-Agile (i.e. using work item cards as records) and still be able to trace the sequence of events and decisions leading to a training solution.
  1. {color:#000;}Structure the project Buyer’s team for fast decision making. Set expectations for all stakeholders to respond in minutes or hours rather than days to avoid them impacting the Lean-Agile project adversely.
  1. {color:#000;}Assign a product owner who is empowered to make decisions for the stakeholders and give direction to the contractor/supplier.
  1. {color:#000;}Assign the Product Owner the responsibility and authority to prioritize all work items in the courseware backlog to match stakeholder priorities and to internally referee any conflicts.
  1. {color:#000;}Assure stakeholders that content and functionality of courseware is still constrained by learning objectives just like with traditional ADDIE approaches.
  1. {color:#000;}Assign stakeholders to attend each synchronous, end-of-iteration demonstration to provide feedback to the development team.
  1. {color:#000;}Assure SMEs, if you are providing any, that SME effort during design and development is the same amount of effort, just distributed differently from traditional projects.
  1. {color:#000;}Inform your stakeholders that the project will use actual courseware increments delivered by the contractor/supplier instead of extensive documentation to have less surprises, less “trust us” and more “show me.”
  1. {color:#000;}Inform your stakeholders that the primary cadence method is the use of fixed-schedule time boxes for each iteration, meaning no re-schedules for demos/reviews. The scope of each iteration is all that varies.
  1. {color:#000;}Let your stakeholders know their engagement will needed during iteration demos every iteration.
  1. {color:#000;}Let your stakeholders know the project specifications will be documented using courseware backlog item cards on a Kanban board instead of a really long MS Word file.
  1. {color:#000;}Prepare stakeholders and SMEs for courseware increments rather than lessons or the entire course at once.
  1. {color:#000;}Inform your stakeholders for a more collaborative and less adversarial relationship with contractors/suppliers because iteration demos allow transparency of progress.
  1. {color:#000;}Inform your stakeholders that quality checks happen earlier and the project will have improved buyer opportunities to give feedback to the contractors/suppliers development team.
  1. {color:#000;}Inform stakeholders that changes can incorporated even late in the project life cycle by swapping out similarly sized lower priorities for higher priorities.
  1. {color:#000;}Prepare stakeholders for higher frequency inspections and evaluations of smaller deliveries, called working courseware increments, than they may have been used on traditional projects. Same effort, distributed differently.

[]A.1.7. Organizational Adjustments to Buyer Process Documentation

Goal: Ensure buyer’s traditional methods do not contribute to Lean-Agile project risk.

  1. {color:#000;}Adopt a uniform taxonomy for Lean-Agile so that all stakeholders/customers and contractor/supplier staff are aligned in communication. See the glossary of this book as a starting point.

  1. {color:#000;}Give those involved new performance expectations (“Help us revise our processes to use Lean-Agile.” “We will support not meeting existing process steps.”)
  1. {color:#000;}Review traditional processes with the contractor/supplier before award to identify the scope of change to existing buyer-internal processes.
  1. {color:#000;}Evaluate using a Lean-Agile set of processes to be used for Lean-Agile projects.

[]A.1.8. Execute Strategy

  1. {color:#000;}If you are a Government buyer, consider a draft request for proposal (RFP).
  1. {color:#000;}If you are a Government buyer, issue a RFP.
  1. {color:#000;}Select the right contractor/supplier.
  1. {color:#000;}Award the contract.
  1. {color:#000;}Finalize your Lean-Agile aligned QASP.
  1. {color:#000;}Conduct a post-award kick-off meeting with the buyer team and implement transition period, if applicable.

|={color:#000;background:rgba(0, 0, 0, 0);}. Tip|<{color:#000;background:rgba(0, 0, 0, 0);}. Consider tasks on this list for your post-award implementation/transition.|

[]A.1.9. Performance Management

  1. {color:#000;}Build and manage the relationship with the contractor/supplier.
  1. {color:#000;}Assess performance.
  1. {color:#000;}Train buyer project manager in Lean and Agile as applied to training.
  1. {color:#000;}Monitor the contractor/supplier Kanban board snapshots for minimal WIP inventory, to help ensure your desired changes can be negotiated for swap-out without minimal rework.
  1. {color:#000;}Conduct a buyer-internal-only stakeholder meeting after the first iteration demo/review because everyone’s expectations will have become even clearer after the first iteration demo conversations as you inspect and evaluate one courseware increment.
  1. {color:#000;}If you are a Government buyer, some of the contractor’s performance metrics artifacts will be the work item cards on the Kanban board.
<{background:rgba(0, 0, 0, 0);}.
p((((((((((={color:#000;}. * Caution*
<{background:rgba(0, 0, 0, 0);}.
p((((((<{color:#000;}. [* Insistence on reformatting everything from the Kanban to other formats is a form of Lean waste. Look for ways to incorporate Kanban board data. Consider periodic photos of the board as record-keeping artifacts.*]
  1. {color:#000;}Restrict scope changes to the beginning of each iteration as the contractor/supplier conducts their iteration planning meeting. Do not expect scope changes in the middle of a fixed iteration.
  1. {color:#000;}Join the contractor/supplier iteration retrospective to identify lessons learned at the end of every iteration.
  1. {color:#000;}Join the contractor/supplier iteration planning meetings to monitor how well the development team feeds forward lessons learned for continual improvement.
Caution As the customer, the buyer should not take over these contractor/supplier meetings when there to observe and become informed of the team’s progress. Leave the development team the time to conduct their meeting. The Product Owner, Lean-Agile Coach and the Development team are responsible for these meetings. Anyone else is an observer. If you attend, shift your thinking from oversight to insight. Hold your comments and questions until after the team breaks and goes back to work to avoid meeting waste. Remember, Lean-Agile emphasizes partnering with the contractor/supplier.*]
Tip*][_ Look at the development team task list for continual improvement tasks to see how much “managing continuous improvement” varies with Lean-Agile instead of the traditional waterfall approach._]

[]Appendix B: Lean-Agile Job Tasks for the Producers[]

[]B.1. Job Tasks for Lean-Agile Development Teams

This section is aimed at training professionals. To better articulate our discoveries about what was really needed on the job for people transitioning to Lean-Agile, we performed a task analysis and developed the following Task List of things people need to be able to do using Lean-Agile on courseware product development projects.

You may need some of these, and others that are not listed. Use this as a start point and adapt for your unique circumstances. You will have to add the training design and development tasks yourself. This is only the Lean-Agile tasks that supplement the training design and development tasks.

Lean-Agile Task List for Learning Experience Development

[]B.1.1. Conduct High-Level Planning Tasks

High-level planning applies to the entire project, not to only one iteration.

  1. {color:#000;}Gather information.

a.Review existing information.

b.Review any existing analysis artifacts.

  1. {color:#000;}Make workflow transparent and visual.

a.Define your workflow steps needed to produce learning experiences.

<{background:rgba(0, 0, 0, 0);}.
p((((((((((={color:#000;}. * Note*
<{background:rgba(0, 0, 0, 0);}.
p((((((((((<{color:#000;}. [_ Lean-Agile does not dictate your process steps._]

b.Create a Kanban board with these process steps as columns.

c.Define the process steps or workflow on the board.

d.Define “done” for each process step.

e.Post done definitions under each process step column on the board.

f.Apply work in process (WIP) limits to reduce cycle time.

  1. {color:#000;}Orient team members.

a.Clarify Lean-Agile modifications to normal roles in training development.

b.Use generic Lean-Agile terms rather than from a specific framework.

c.Conduct Lean-Agile training as needed.

d.Clarify expectations for built-in quality of work.

e.Ensure all involved understand what “done” means for each process step.

f.Identify required conventions, standards or guidelines.

g.Select more stringent criteria for quality as the team matures.

h.Establish team ground rules.

The following is not comprehensive, but only an example of a few: (a) Everybody works, (b) Pull the next highest priority work item card, © Use Kanban card avatar to visually signal ownership to others on the team, (d) Share stakeholder feedback with the team.

i.Clarify job specialization expectations (instructional/LX designers versus media for example).

  1. {color:#000;}Determine if design can be done during iterations or if a CDD (course design document) is required.

  1. {color:#000;}Generate requirements (crucial for games or simulations).
  1. {color:#000;}Conduct high-level design iterations (if you can do them in your circumstances).

a.Determine how many iterations time and budget allow.

b.Generate learning objectives (LOs) from the analysis task list.

c.Determine evaluation strategy for those LOs.

d.Get buy-in from SMEs and stakeholders on LO scope and sequencing.

e.Generate initial ideas for instructional strategies.

f.Collaborate with SMEs/Stakeholders about instructional strategies.

g.Create the CDD (courseware design document) (just like in ADDIE) if prototype primacy is not agreed upon with customer.

h.Complete the test-case checklist for CDDs as the definition of “done”.

i.Create the game or simulation design document (if applicable).

j.Rapidly prototype.

  1. {color:#000;}Determine how many iterations time and budget allow.

  1. {color:#000;}Assume that first decisions will not be valid and that early design work will be discarded.
  1. {color:#000;}Create prototypes over excessive documentation (to the degree the customer allows) to help stakeholders visualize the end state better.
  1. {color:#000;}Prepare only representative content and call it a “functional prototype” not a “sample lesson prototype” so that the focus is on the functionality rather than the content.
  1. {color:#000;}Demonstrate the prototype in a synchronous (group-paced) meeting to get quick feedback.
  1. {color:#000;}Complete a prototype checklist with specific test cases as the definition of “done” for consistent built-in quality.
  1. {color:#000;}Identify the initial courseware backlog.

a.Decompose the training product (courseware) into a product backlog.

b.Prepare Kanban cards for each work item.

c.Prioritize based on customer value, risk and opportunity.

d.Articulate why prioritizing high risk work items first reduces risk for all involved.

e.Order the backlog so the highest priority work item cards are at the top of the stack.

  1. {color:#000;}Setup development tools (non-recurring).

[]B.1.2. Tasks to Plan the Iteration

The following producer job tasks plan only the one iteration.

  1. {color:#000;}Apply lessons from prior iterations.

  1. {color:#000;}Use two-week fixed iterations to limit risk and complexity.
  1. {color:#000;}Commit to an iteration goal (scope) and include it as a deliverable card type.
  1. {color:#000;}Create the Iteration Backlog.

a.Pull work item cards from the courseware product backlog to this iteration’s backlog.

b.Add any required tasks to the backlog.

c.For each learning objective, add its components as work item cards.

d.Decompose the effort into similarly-sized work items (LOs rather than lessons).

e.Break down high-level work items further into items small enough to be completed in a single iteration.

f.Estimate the work items sizes, if not using historical metrics.

  1. {color:#000;}Complete the iteration planning checklist.

[]B.1.3. Tasks to Conduct this Iteration

The following producer job tasks conduct only the one iteration.

  1. {color:#000;}Apply Kanban.

a.Apply Lean visual management for transparency.

b.Pull Kanban cards from upstream, rather than push them downstream to get appropriate buffers between process steps, which in turn reduce bottlenecks (see Theory of Constraints for why).

  1. {color:#000;}Synchronize the team every day in a daily standup meeting.

a.Conduct daily standup synchronization meetings (use checklist if new).

b.Use visual feedback from the Kanban board to improve.

c.Establish a regular cadence (aka takt time, tempo, team pace).

d.Match the cadence to the team’s capability to reliably deliver the working product increments at a dependable speed.[78]

e.Reduce coordination costs (time box meetings to prevent waste).

f.Collaborate during an iteration.

g.Complete work items you pulled.

  1. {color:#000;}Inspect and adapt toward the iteration goal.

a.Meet the expected cadence.

b.Inspect artifacts and progress to detect issues at the point of work (gemba).

c.Optimize for team throughput over individual performance (this takes a lot of persuading to some people initially).

d.Continually implement the highest-priority work items from the backlog to ensure what you deliver will be the highest value possible even if some of the original backlog work items are different by the end (this only works if your customer contract allows for it).

e.Prevent changes during a fixed iteration that endanger the iteration goal.

f.Reduce cycle time—Use it as a good approximation of effectiveness

  1. {color:#000;}Use small batch sizes and one-piece flow.

  1. {color:#000;}Apply the Theory of Constraints to remove bottlenecks.
  1. {color:#000;}Free up resources to fix the bottleneck.
  1. {color:#000;}Remove blockages actively.
  1. {color:#000;}Reduce transaction costs to facilitate doing smaller batches cost effectively.
  1. {color:#000;}Apply WIP limits to minimize context switching waste.

g.Lean out waste.

  1. {color:#000;}Reduce setup time.

  1. {color:#000;}Identify anything that (a) delays the start of work, (b) interrupts work, © requires ramp up to “full speed”, or (d) is similar to or identical to another task.
  1. {color:#000;}Offload interruptive and delaying tasks if possible.
  1. {color:#000;}Streamline or automate tasks.
  1. {color:#000;}Use Lean problem-solving tools.

h.Standardize to reduce complexity, reduce task variance (i.e. Standard Work).

i.Focus efforts on variables that regulate throughput per Little’s Law: WIP and cycle time.

j.Apply Lean principles.

k.Increase the degree of employee participation in improvement and increase team self-management through information sharing.

  1. {color:#000;}Analyze components (this may just be requirements generation at the component level).

  1. {color:#000;}Design components (elements of ADDIE still apply here, do one learning objective at a time for one-piece flow).
  1. {color:#000;}Develop components (elements of ADDIE still apply here, done one learning objective at a time for one-piece flow).
  1. {color:#000;}Test components.

a.Perform automated tests.

  1. {color:#000;}Perform spell check.

  1. {color:#000;}Find more ways to automate testing.

b.Perform manual tests

  1. {color:#000;}Complete checklist for accessibility compliance (Section 508).

  1. {color:#000;}Confirm text content conforms to the project style guide.
  1. {color:#000;}Complete test case checklist to ensure the User Interface (UI) works as intended.
  1. {color:#000;}Complete functionality and content checklist for UI tutorial and help screens.
  1. {color:#000;}Complete functionality and content checklist for xAPI (if applicable).
  1. {color:#000;}Complete test case checklist for SCORM (if applicable).
  1. {color:#000;}Confirm level of interactivity is met (if contractually required).
  1. {color:#000;}Complete content for instructional/LX design validation.
  1. {color:#000;}Complete functionality and content checklist for learner assessment test bank.
  1. {color:#000;}Complete content checklist for storyboards/scripts.
  1. {color:#000;}Playtest game and simulation components.
  1. {color:#000;}Integrate components in to the courseware build, authoring tool or learning content management system (depends on project).

[]B.1.4. Tasks to Deliver the Iteration Results

  1. {color:#000;}Deliver this iteration’s results.

a.Deliver a working product increment each iteration.

b.Check customer satisfaction with the product increment.

c.Facilitate an iteration review/demo.

  1. {color:#000;}Provide the working increment to stakeholders.

  1. {color:#000;}Review the iteration goal.
  1. {color:#000;}Discuss any variance from the iteration goal, root cause and recovery plan.
  1. {color:#000;}Ensure the team demonstrates the work it has “done” and listens for feedback.
  1. {color:#000;}Capture review minutes and share with the stakeholders.

d.Confirm how close we are to the customer’s intent.

e.Use different colored Kanban cards for defects or change requests (CRs) – we use orange.

[]B.1.5. Tasks to Conduct the Iteration Retrospective

The following producer job tasks conduct the retrospective for this iteration.

  1. {color:#000;}Conduct this iteration’s retrospective.

a.Inspect how the last iteration went with regards to people, relationships, process and tools.

b.Capture iteration lessons learned.

c.Develop a plan for applying improvements to the team’s process.

d.Apply Kaizen, continual improvement (get a little better each iteration).

e.Discuss work in process (WIP) limits.

f.Measure change requests (CRs) and defects.

  1. {color:#000;}Measure CRs/defects generated (rate per iteration).

  1. {color:#000;}Measure CRs/defects resolved (rate per iteration).
  1. {color:#000;}Measure CRs/defects open today (quality).
  1. {color:#000;}Measure CRs/defects that escaped this iteration.

g.Measure unfinished work items during the iteration.

h.Measure productivity per student seat hour (at the end of the project).

i.Complete the iteration retrospective checklist.

  1. {color:#000;}Apply review/demo feedback to improve and optimize the team’s performance.

a.Apply lessons learned from prior iterations to next iterations.

b.Add new work items (new scope demand).

c.Transfer unfinished work items to the courseware backlog for another iteration.

d.Optimize for the entire development process level over individual efficiencies.

So in summary, there is a proposed current task list. This can help you train your teams to be able to do these tasks. They probably won’t know the Theory of Constraints or be able to apply it on their first iteration, but as the team matures, they need to know how to apply this theory too.

[]Appendix C: Markup Technologies

Lets take a moment to review an apparently trailing-edge technology in some domains that is actually leading edge in the training industry.

General text markup is a trailing-edge technology that has been with us since the printing press and moveable type. Human languages are still a primary means of communicating ideas, and even though written text is a trailing edge technology as a whole, when taking the perspective of components and subcomponents of courseware, distinctions in these supporting text technologies can be viewed as leading-edge status for the training domain, although they are already mainstream technologies in the technical writing and publishing domains.

By text markup we mean information formally distinct from the character sequence of the text. Original markup started with handwritten notes to a typesetter on a piece of paper. Computer markup includes fonts, font sizes, emphasis like bold and italic, et cetera and has many methods that are proprietary and owned by particular authoring tool vendors.

To avoid proprietary lock in to a particular authoring tool vendor, some organizations have moved to markup text in more open markup schemes for encoding text in computers. Further distinctions of text markup technologies have included IBM’s generalized markup language (GML)] [79] in 1969, standardized general markup language (SGML) [80] in 1980, to hypertext markup language (HTML) in 1990.

Text markup began to include semantic markup with eXtensible Markup Language (XML) [81] in the mid 1990s, to Javascript object notation (JSON) [82] in 2001, and to HTML5 [83] in 2014.

By semantic markup, we mean using text markup to indicate the meaning of bits of content rather than the primary focus of earlier text markup on defining its presentation or look. The separation of concerns, of content from layout formatting, helps to reduce training developer effort by automating the layout and presentation formatting.

The refinements in text markup from presentation to semantic have made this technology more leading edge. Other domains have refined their use of semantic-oriented text markup for decades now, making it mainstream for them. For the training domain, however, the use of semantic-oriented text markup is still leading edge.

Let’s consider a few simple concrete examples. Each of these examples shows a different technology tweaking text-based markup. In the end, note that all these examples are encoded in text. [Plain, old, long-lasting, *]text[.*] For those of you wondering how the presentation formatting is done, for now let’s just leave it at a high level. Presentation and layout are accomplished using style sheets. This separation of content and style is the main idea. If you don’t need or want more detail than that, then skip to the end of this paragraph. If you’re still reading, I will simply point to other places on the web that you can find out more. HTML uses Cascading Style Sheets (CSS).[84] Recent inventions to reduce the effort for CSS faster tend to interest software developers more than a wider audience, so I will leave out some of the latest front end shortcuts to CSS in this overview. XML also uses style sheets, but the technology is a little different. XML uses eXtensible Stylesheet Language (XSL).[85] JSON, markdown, and asciidoc all use specific applications or general programming languages to get back to another format, such as HTML5, which then uses CSS. Okay, that is enough alphabet soup for now. Let’s look at an example.

The following example is HTML5 markup. The comment line simply states which parts are markup. By the way, HTML5 involves a whole set of technologies that is beyond the scope of introduction.


This is the title

This is the content.

  1. {color:#000;}HTML5 includes a semantic element called

  1. {color:#000;}The title is indicated by the


The following is an XML example just to illustrate. XML differs in that you can invent your own tags or elements. See how we have wrapped the content in meaningful elements in items 1-2 below. Callout 3 simply shows a comment line.


(2) (3)
This is the title
This is the content.

  1. {color:#000;}Semantic container for course.

  1. {color:#000;}Semantic container for lesson.
  1. {color:#000;}Comment Line is not rendered to the learners.

The following is a JSON (www.json.org) example without the comment (to avoid the debate about whether or not to use comments in JSON for this example).


“course”: {
“lesson”: {
“title”: “This is the title”,
“para”: “This is the content.”

Okay, that should give you a picture about markup technology.

Now here is the same example in markdown, [86] and asciidoc.[87] Both markdown and asciidoc are also getting popular and can be converted into courseware. Here is an example of markdown, and another of asciidoc.


#This is the title (1)
This is the content.

  1. {color:#000;}The title level is indicated by the ‘#’ symbol in markdown.


:lesson: (1)
// Here is a comment (2)
=This is the title (3)
This is the content.

  1. {color:#000;}Semantic attribute for lesson.

  1. {color:#000;}Comments are allowed in asciidoc.
  1. {color:#000;}The title is identified by the ‘=’ symbol in asciidoc.

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. The similarities between markdown and asciidoc are great if creating low complexity content with paragraphs and lists. Markdown works great for blogging in text and automatically converting to HTML for posting. However, when you have professional level technical documentation requirements, then asciidoc is a stronger solution because it handles more difficult publishing features that markdown cannot.|

Notice how little markup exists in the last example of markdown and asciidoc. The ease of asciidoc and markdown is helping their popularity among software developers and others. Markdown continues to spread beyond coders and bloggers because its ecosystem now includes full pipeline tools. For more details, search the web for pandoc,[88] multimarkdown,[89] LaTeX [90] and Apache™ FOP [91] (Formatting Objects Processor) it is possible to get markdown and asciidoc to HTML or PDF [92] in an automated way, saving time and, therefore, money.

Asciidoc is like markdown because it has minimal syntax compared to HTML5, XML or JSON. It is also exceptionally easy for humans to read. It can be used to automatically and quickly convert text-based content encoded in asciidoc syntax into various formats including, HTML, PDF, ePub,[93] plain text, etc. Because it is free and open source software, using asciidoc can help lower initial costs, and it can help lower recurring labor costs because its automated conversion tool chain reduces cycle time.

If you have people on staff who already have to write code in XSLT,[94] JavaScript,[95] CSS, or other programming languages, they are already familiar with using text editors to edit content, most often on a daily basis.

We have used asciidoc to encode text-based content for fast courseware storyboards. We have used this text-based encoding method for courseware design guides. We have used this text-based encoding for courseware instructor guides and student guides. It works well for technical whitepapers, and we used asciidoc to write this book.

Like more complex XML tool chains such as the Darwin Information Typing Architecture (DITA) [96] XML that IBM originally built and then open sourced, asciidoc lets us break content into smaller chunks for reuse of text-based content components similar to how software developers reuse object-oriented classes between products. Such reuse can save costs and reduce risk.

We can use and reuse text-based content documents rather than what-you-see-is-what-you-get (WYSIWYG), so content can be version controlled in free and open source tools like git,[97] just like text-based software code and included by reference just like software modules/libraries in a software application. This is one of the key benefits of text-based training content.

We have also used XML encoding for various MIL standards, such as S1000D and MIL STD 40051. However, XML encoding requires specialized commercial apps to validate against the schema. These commercial apps can be expensive and complex to initially learn. We have discovered that markdown and asciidoc have lower non-recurring licensing costs, and that both reduce cycle time by reducing complexity, saving labor costs during content creation.

To summarize, using text-based content encoded in

  1. {color:#000;}markdown – for semantically simple content

  1. {color:#000;}asciidoc – for semantically complicated and modular content

… allows us low startup costs for experimentation, fast and efficient version control of content, reuse during development and offers a more efficient workflow with a focus on content without constant regard to layout formatting until later in the workflow. These text-based encoding schemes also allow us to build training-related content in a modular way and aggregate into a single large collection or SCORM package and then reuse content using a feature called include. For developers out there, this feature of asciidoc is much like the require statement in ruby programming, the import statement in java programming and the #include statement in C++ programming.

We have found these simple text encoding schemes (markdown and asciidoc) to be faster for content development than HTML or XML directly for the text content. Also, as we have built automatic metric collection and quality assurance checking for courseware, we used the content encoded semantically so that we can algorithmically query the content and compare the actual to the design should.

Text-based encoding of training content also offers buyers that need long shelf-life courseware a way of maintaining source materials over decades without dramatic encoding conversion projects. It also offers buyers the ability to have other contractors/suppliers perform updates and maintenance to courseware made by someone else.

This technology offers many training buyers and producers advantages. This technology is mainstream in other domains. And yet, curiously, it still seems to be leading edge in the training industry. Not many training organizations seem to use it yet. Why this matters:

  1. {color:#000;}Lean suggests saving transaction costs by using semantic-oriented text markup technology.

  1. {color:#000;}Lean also suggests that using a modular approach with any semantic-oriented text markup allows us to work on small batches, reducing cycle time.
  1. {color:#000;}Agile lets us focus on working courseware earlier in the production lifecycle than we did with sequential ADDIE waterfall approaches.
  1. {color:#000;}Semantic-oriented text markup technology can help the training domain especially for eLearning and performance support efforts.

[]Appendix D: Further Reading[]

[]D.1. Further Reading

For additional reading consider the following:

  1. {color:#000;}A Leader’s Framework for Decision Making, by David J. Snowden and Mary E. Boone, Harvard Business Review, November 2007

  1. {color:#000;}Antifragile – Things that Gain from Disorder by Nassim Nicholas Taleb
  1. {color:#000;}CMMI for Development, Second Edition, by Mary Chrissis, Mike Konrad, and Sandy Schrum.
  1. {color:#000;}In Search of the Elusive ADDIE Model, by Michael Molenda of Indiana University, published in Performance Improvement, May/June 2003 by ISPI.org
  1. {color:#000;}Kanban: Successful Evolutionary Change for Your Technology Business, by David J. Anderson, 2010
  1. {color:#000;}Lean Software Development: An Agile Toolkit, by Mary Poppendieck and Tom Poppendieck, Addison-Wesley, 2003
  1. {color:#000;}Lean Product Development: Making waste transparent, by Christoph Bauch Ph.D. Massachusetts Institute of Technology 2004
  1. {color:#000;}Lean for Systems Engineering, by Bohdan W. Oppenheim, 2011
  1. {color:#000;}Lean Thinking, by James P. Womack and Daniel T. Jones, 1996
  1. {color:#000;}Leaving ADDIE for SAM, by Michael Allen, 2012
  1. {color:#000;}Scrum: The Art of Doing Twice the Work in Half the Time, by Jeff Sutherland (Sep 2014)
  1. {color:#000;}The Checklist Manifesto: How to Get Things Right, by Atul Gawande, 2011
  1. {color:#000;}The Complete Guide to Simulations and Serious Games: How the Most Valuable Content Will be Created in the Age Beyond Gutenberg to Google, by Clark Aldrich, October 2009 (http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470462736.html)
  1. {color:#000;}The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt, Jeff Cox and David Whitford (Jun 2012)
  1. {color:#000;}The Machine That Changed the World by James P. Womack, Daniel Roos, and Daniel T. Jones, 1990
  1. {color:#000;}The Principles of Product Development Flow: Second Generation Lean Product Development by Donald G. Reinertsen (May 29, 2009)
  1. {color:#000;}The Thinking Effect: Rethinking Thinking to Create Great Leaders and the New Value Worker, by Michael Vaughan, 2013
  1. {color:#000;}Thinking in Systems, by Donella Meadows, 2008.
  1. {color:#000;}Velocity: Combining Lean, Six Sigma and the Theory of Constraints to Achieve Breakthrough Performance – A Business Novel, by Dee Jacob, Suzan Bergland, Jeff Cox (Dec 2009)
  1. {color:#000;}Characteristics of Agile Organizations, by Ray Arell, Jens Coldewey, Israel Gat, Jorgen Hesselberg, 2012, http://www.agilealliance.org
  1. {color:#000;}SAFeLSE, http://safelse.com
  1. {color:#000;}SAFe, http://www.scaledagileframework.com
  1. {color:#000;}Presentation slides, Glen Alleman – Niwot Ridge, LLC, http://www.acq.osd.mil/evm/resources/AgileMtgSlides_andAgenda/Alleman_EV+Agile=Success%20(V3).pdf
  1. {color:#000;}The Process Mind: New Thoughtware for Designing Your Business on Purpose, by Philip Kirby, Productivity Press © 2015
  1. {color:#000;}The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, by Eric Ries
  1. {color:#000;}Adapting Configuration Management for Agile Teams: Balancing Sustainability and Speed, by Mario E. Moreira, John Wiley & Sons © 2010

[]19. Glossary and Acronym List

This glossary and acronym list helps make clear the meaning of the words and acronyms used in this book. There are some terms where different practitioners mean different things.

[]19.1. A


Analysis, Design, Development, Implementation, Evaluation


Agile product development is based on Agile principles. It is a group of development and project management methods derived from Agile principles, and based on iterative and incremental development. The solution evolves through collaboration between self-organizing, cross-functional teams.[98] It promotes adaptive planning and encourages rapid and flexible response to change.[98] The Agile Manifesto introduced the term for software development in 2001. There are many methods that can be Agile, including Scrum, Kanban, and Scrumban. Agile emphasizes teamwork, frequent deliveries of working courseware, close customer collaboration, and responding quickly to change.

agile coach

The person who implements and guides the Agile processes in the project. They often serve as a facilitator and their authority is typically indirect.

[]19.2. B


The backlog is the prioritized stack of requirements for a course, simulation, or game, expressed as a prioritized list of product backlog work items. It is often visibly located on the left side of a Kanban board with the backlog work items prioritized with the highest priority work item card at the top. Requirements in courseware include (a) learner experience objectives, (b) interaction functionality requirements, and © instructional content requirements. In Scrum, the product owner is responsible for the backlog. During an iteration planning meeting, backlog items are moved from the overall product backlog into the iteration backlog.

big requirements up-front (BRUF)

A term from software development that applies to the waterfall approach way of trying to gather and completely document all requirements in one big effort early on.

black box

In science, computing, and engineering, a black box is a device, system or object which can be viewed in terms of its inputs and outputs (or transfer characteristics), without any knowledge of its internal workings. Its implementation is “opaque” (black). Almost anything might be referred to as a black box: a transistor, algorithm, or the human brain.[99] The opposite of a black box is a system where the inner components or logic are available for inspection, which is most commonly referred to as a white box (sometimes also known as a “clear box” or a “glass box”).[99]


A constraint or traffic jam in the process slowing or piling up work items. Any limit to process capacity versus the demand amount. See the Theory of Constraints for more detail.


A term from software development that applies more and more to training development. In software, as a verb, the term build refers to integrating and compiling the source code files into an executable application. For training products, it is the published courseware, whether a SCORM package, an instructor package or the content and application files for mobile learning. Each build should be uniquely identified to testing and reporting defects. Complex courseware may include many files that may be combined in different ways depending on the delivery method. Blended ILT components, eLearning and mLearning all tend to include software code and media components in the build.

burndown chart

A burn down chart is a graphical representation of work left to do versus time.[100] The outstanding work (or backlog) is often on the vertical axis, with time along the horizontal.[100] That is, it is a run chart of outstanding work.[100] It is useful for predicting when all of the work will be completed.[100] Outstanding work can be represented in terms of either time or story points.[100] The iteration burndown chart shows iteration progress, while a product burndown chart shows progress of the courseware as a whole.

[]19.3. C


Customer-required ability of the system that provides value to the end user


Courseware Design Document

change request

A formalized way of proposing a change to the product or service, sometimes needing approval before implementation


Capability Maturity Model Integration

context switching

The amount of time required to change to a different work item. It is the waste from multitasking. It is the act of turning a worker’s attention and mental focus from one task to another, rather than having them do one thing at a time until it is complete. Sequential processing gets you faster results on average, and the longer it takes to switch tasks, the bigger the waste penalty you pay for multitasking. Research backs up the assertion that context switching increases processing time and error rate. Task recovery time includes how long a worker takes to resume the task at the pace at which they left off before the interruption. Mitigation strategies include limiting work in-process and using the Pomodoro technique.[101]


A bottleneck or traffic jam in the process slowing or piling up work items. Any limit to process capacity. See the Theory of Constraints for more detail.

continuous flow

Team members involved in courseware work item processing orient around each work item rather than their station/process step perspective. The goal of continuous flow is for each work items to flow from one process step to the other with little to no waiting. It is facilitated by pull assignments and one-piece processing.

continual improvement

An ongoing effort to improve products, services, or processes. These efforts can seek “incremental” improvement over time or “breakthrough” improvement all at once. Delivery (customer valued) processes are constantly evaluated and improved in the light of their efficiency, effectiveness and flexibility.[102]


In software engineering, coupling is the manner and degree of interdependence between software modules; a measure of how closely connected two routines or modules are; the strength of the relationships between modules.[103] Coupling is usually contrasted with cohesion.[103] Low coupling often correlates with high cohesion, and vice versa.[103] Low coupling is often a sign of a well-structured computer system and a good design.[103]


A complete integrated series of lessons which are identified by a common title and/or number.[104] A related grouping of lessons. Usually on a time constraint.[104] In our usage, we mean workplace training.


The combination of content and player software needed to deliver a course for mLearning, eLearning, or blended instructor-led training (e.g. including interactive components). The content includes instructional text, media, test items, accessibility content for equivalent facilitation, and instructional feedback. The player contains the software code that provides the functionality to navigate through the content, grade learner assessments, and user controls for audio and video playback, as well as make selections in branching simulation scenarios and report SCORM data (eLearning) or xAPI data (mLearning). Depending on the complexity of the game or simulation components, the courseware may also include system models and responsive models to provide dynamic interactions.

courseware design document (CDD)

A plan, like a blueprint for a house under construction, that outlines the intended learning experience scoped at the course level. The CDD describes the effective skill and knowledge interventions to close those gaps in human performance identified in the training analysis. The CDD includes a summary of the overall instructional purpose of the courseware, a description of the target population, the set of learning objectives that are in scope, a courseware outline, the flow or sequencing of the learning objectives, the assessment strategy, a description of the instructional approaches for the content and target population, descriptions of the planned interactive experiences at the course-level granularity, the rationale for how the planned interactions provided the learning experiences that achieve the learning objectives, media specifications at course-level detail (including their instructional purpose), and references to expected requirements (e.g. accessibility, video codecs, style guides, etc.).

cumulative flow diagram (CFD)

A diagram that shows the number of work items in each process phase (design, development, etc.) on a given day and tracks it over time on a stacked area chart in Excel. If a phase shows a wide area, it indicates there is a lot of work in-process in that particular phase. If this continues, it indicates something is interrupting the flow.


In the context of vocational training, not education, curriculum is the aggregation of courses in a particular workplace. A curriculum is statements of intention using learning outcomes to define overall goals and specific outcomes of a learning program. Courses are often prescriptively sequenced to aid learning the knowledge and skills more easily.

Cynefin Framework

A conceptual framework developed by David Snowden intended to describe a perspective on the evolutionary nature of complex systems, including their inherent uncertainty.[105] It offers approaches for various conditions and is based on complex adaptive systems theory.[105]

[]19.4. D

daily scrum (aka daily standup)

A 15-minute daily meeting for each team member to answer what they’ve done, what they’ll do, and reveal any obstacles. The Team Lead or Agile Coach ensures that participants call sidebar meetings for any discussions that go outside of these constraints.


A deficiency that results in (or would result in) improper operation, impaired performance, or cause rework to an existing work product (courseware). Defects are waste because we have to pay to make them. Then we have to pay again to fix them. Also called technical debt by software developers.


The desire of customers, reviewers, testers or others for a particular service. An order. Sometimes called a requirement. Value demand is a demand type that customers are willing to pay for. Defect demand is another type of demand, but is waste. New or change demand is a scope change.

Deming cycle

See Plan-Do-Check-Adjust cycle, or PDCA.


(1) an instructional/LX design is a systematic approach for a learning experience or performance support product development that ensures that specific performance goals are accomplished. In Lean-Agile, it is an iterative process that requires on-going evaluation and feedback. The design is the blueprint and precedes development. (2) a game or simulation design is a blueprint for the game or simulation.

design document

A document that spells out the intended courseware. It describes objectives, strategies and approaches.

design intent

The planned purpose, functionality and content of a finished courseware product that meets its requirements. The design intent is often captured in CDD or storyboard artifacts and tends to be separated from the final courseware, making maintenance more difficult.


Do It Yourself

[]19.5. E

enabling learning objective (ELO)

A learning objective that students must attain in order to accomplish a terminal learning objective.[106] ELOs are developed from one or more tasks in the courseware task list. The learner may accomplish ELOs at any point in the courseware after receiving appropriate training.


A big work item where the effort is too big to finish in a single iteration. This work item is still too big and should be broken down into smaller chunks that can be completed within a single iteration. There is no consensus on exactly how big an epic is. The word is intended to convey bigger than normal. The origins of this term are from software Agile projects.


The systematic assessment of a course or tool to insure that it meets pre-determined objectives. Also, the systematic assessment of a course or tool to insure that it meets objectives in as efficient and effective a manner as possible.


Earned Value Management

[]19.6. F


Part of a capability which can be completed within an incremental release (e.g. 3–6 months)


Corrective feedback is a frequent practice in learning.[107] It typically involves a student receiving either formal or informal feedback on his or her performance on various tasks by a teacher or peer(s).[107] Computers can also be programmed to provide corrective feedback.[107] Feedback applies in branching stories, games and simulations too with feedback based on state changes in the story, game or simulation. A game involves human interaction with a user interface to generate visual feedback.[108] If an incorrect action is tried then the game should give some form of feedback to help guide the player.[109]

[]19.7. G

Getting Things Done (GTD)

A book by David Allen that focuses on personal effectiveness strategies


The scale of level of detail present in a set of content, data or other phenomenon. For training, it is the chunk size we are talking about, for example course, lesson, learning objective.

[]19.8. I


Interactive Multimedia Instruction, as described by the US DoD MIL-HDBK-29612-3A. It used to be called computer-based training. It is also called web-based training and eLearning.


When using Agile, user stories are used as inchstones for earned value reporting to reinforce value of working [courseware] over completion of tasks. Inchstones are not IMS tasks and therefore have no budget, scope or schedule assigned to them. Use story point weightings to calculate the earned value contribution of each completed story. Use of story point weights is the “defined value” used to calculate milestone percent complete.[29]


An increment is a working partial product that helps the customer see the progress to date. One increment is developed each iteration. The primary measure of progress rather than extensive documentation.


An organized examination or formal evaluation exercise.[110] Involves the measurements, tests, and gauges applied to certain characteristics in regard to an object or activity.[110] The results are usually compared to specified requirements and standards for determining whether the item or activity is in line with these targets, often with a Standard Inspection Procedure in place to ensure consistent checking.[110] Inspections are usually non-destructive.[110] Inspections may be accomplished manually or automatically.[110]


Integration refers to the merging of lower-level items (subsystem or product) into the next higher level (system), and the testing that shows that the next higher-level item functions as expected and is ready for verification and validation. In training, examples include audio assets, video assets and interactions that are integrated into the courseware. For complex eLearning, that may include game engine content, where integration includes adding all of the models into the scene and adding animations and code.


A kind of action that occurs as two or more objects have an effect upon one another. The idea of a two-way effect is essential in the concept of interaction, as opposed to a one-way causal effect. A closely related term is interconnectivity, which deals with the interactions of interactions within systems: combinations of many simple interactions can lead to surprising emergent phenomena.[111] Interactive experiences are time dependent. Interactive courseware means that learners influence or have an effect on each other or with components of the courseware facilitated by computers or instructors in blended courseware. The system boundaries must include at least two interactive agents. Human interaction may take place between any of the human senses, for example conversation, eye contact, voice tone and body language. When one actor is a customer, these interactions are also called touchpoints in the interaction design literature. Examples may include student practice exercises, narrative scenarios, student response systems, entries into a computer-based component, link traversal, using screen widgets, progressive reveal tabs and media, deterministic branching decision scenarios with navigation based on learner decisions, test items, knowledge checks, spreadsheets, interactive video, games and simulations (both deterministic and probabilistic/stochastic). Computer-based interactions typically included software-driven functionality based on user inputs (e.g. JavaScript in HTML5). Interactivity requirements often include categories or levels with examples of each type. The higher the interaction functionality, the larger its impact on the courseware project budget.

interaction design

The practice of designing interactive digital products, environments, systems, and services.[112] Interaction design also has an interest in form but its main focus is on behavior.[112] It involves synthesizing and imagining things as they might be, more so than focusing on how things are.[112] Interactions are sequenced chronologically as time moves along. Interaction design in learning experience development is about active learner behaviors between two or more agents—People, computer algorithm, or other entities or objects capable of responding.


Iteration is the act of repeating a process, either to generate a unbounded sequence of outcomes, or with the aim of approaching a desired goal, target or result.[113] Each repetition of the process is also called an “iteration”, and the results of one iteration are used as the starting point for the next iteration.[113] Scrum is an iterative and incremental agile software development methodology for managing product development.[114] It defines a flexible, holistic product development strategy where a development team works as a unit to reach a common goal, challenges assumptions of the traditional, sequential approach to product development, and enables teams to self-organize by encouraging physical co-location or close online collaboration of all team members, as well as daily face-to-face communication among all team members and disciplines in the project.[114] The development team produces a working product increment during each iteration. Working means the increment has instructional content that is ready for review, interaction and user interface functionality is tested and it works, and the increment is potentially ready for students pending incorporation of reviewer feedback. The team starts each iteration with an iteration planning meeting. At the end of each iteration, the team holds an iteration review/demo meeting and an iteration retrospective meeting.


Storage of any type of item, information or document. Inventory waste includes mismatches throughout the process chain, often resulting from imbalanced demand and supply. The mismatch stems from poor understanding of customer needs, irrational forecasting and other root causes. “Partial products” show up even in transactional processes, such as slow reviews of outstanding lessons. A thorough understanding of the sources of variability in the supply chain (including SMEs) will lead to the right mix of reduced inventory levels.

[]19.9. J

job aid

A checklist, procedural guide, decision table, work sheet, algorithm or other aid which aid in task performance.[115] A performance support tool to be used at the work site on the job. It may be a video segment, a checklist, posters, mobile apps, in electronic or other formats. Job aids can be used when the work performance does not require memorization of the performance steps, for example if time is available to reference the job aid.

[]19.10. K


(a) Kanban is a Japanese word that indicates a visual card. (b) In courseware and software, Kanban is an Agile method of Lean development.

Kanban visual card

A Kanban card is a visual signal that triggers an action. It provides a transparent method for viewing work and organizing teams. When watching a team of knowledge workers, it is hard to see the process. It is hard to optimize something you cannot see. Kanban provides more visibility to where work is before it is done.


Kaizen, Japanese for improvement. When used in the business sense and applied to the workplace, kaizen refers to activities that continuously improve all functions and involve all employees from the CEO to the line workers.[116]

key learning point (KLP)

A learning objective can be broken down into its associated key learning points as more manageable sized chunks. The equivalent instructor-centered term is teaching point. The U.S. Navy NAVEDTRA M-130B calls KLPs discussion points instead. Key learning point is the preferred learner-centered term.[117]

Kirkpatrick level

Donald Kirkpatrick’s four levels of evaluating training. A commonly used method of articulating the rigor of evaluation for training courseware. Wikipedia states, [118] “The four levels of Kirkpatrick’s evaluation model are as follows: (Level 1) Reaction – what participants thought and felt about the training; (Level 2) Learning – the resulting increase in knowledge and/or skills, and change in attitudes. This evaluation occurs during the training in the form of either a knowledge demonstration or test; (Level 3) Behavior – transfer of knowledge, skills, and/or attitudes from classroom to the job (change in job behavior due to training program). This evaluation occurs 3–6 months post training while the trainee is performing the job. Evaluation usually occurs through observation; and (Level 4) Results – the final results that occurred because of attendance and participation in a training program (can be monetary, performance-based, etc.).

[]19.11. L


(1) A term used by convention in this book to describe our use of Agile and Lean in learning experiences. We chose to use the term Lean-Agile to emphasize that both Lean and Agile are complementary approaches to our training audience that is largely new to both Lean and Agile. It refers to our combination of the principles and ideas from Agile Scrum, Kanban and Lean and translates it to the knowledge work of learning experience development. (2) The term used by the Scaled Agile Framework to address the combination of Lean and Agile.

learning curve

A plot showing how learning improves with experience. A graphical representation of the increase of learning (vertical axis) with experience (horizontal axis).[119]

learning experience (LX)

Refers to any interaction, course, program, or other experience in which learning takes place, whether it occurs in traditional settings [classrooms] or nontraditional settings, or whether it includes traditional interactions (learning from [facilitators]) or nontraditional interactions (learning through games and interactive software applications). [120] Because students may learn in a wide variety of settings and ways, the term is often used as a more accurate, preferred, or inclusive alternative to terms such as course, for example, that have more limited or conventional connotations.[120] Learning experience may also be used to underscore or reinforce the goal of an [learning] interaction—learning—rather than its location or format (course, program), for example.[120] In our usage, we learning experiences designed to support workplace performance rather than general education. Also see Connie Malamed’s blog entry and rationale about using LX design instead of instructional design.

learning experience (LX) design

Refers to the design of learning experiences. LX borrows from user experience (UX). Some organizations are using LX Design now instead of instructional design. [121]


Learning and Development

learning objective (LO)

A statement in behavioral terms of what is expected of the student in demonstrating the knowledge and skill level necessary. LOs are developed from one or more tasks in the courseware task list. Some organizations further distinguish LOs into terminal LOs and enabling LOs. Some organizations now call LOs performance goals. To avoid any confusion performance goals as used by human resources for performance management, we will use LOs in this book.

lessons learned

Significant knowledge gained during the performance of a process that is identified as either 1.) an adverse effect that others should avoid in the future or 2.) an improvement that others could benefit from. Lessons learned are one type of improvement information, and an outcome of Agile iteration retrospective meetings.


A Learning Management System (LMS) is used to administer one or more courses to one or more learners.


Learning Objective


Learning Experience or Learner Experience

[]19.12. O


The induction, assimilation, initial training, paperwork, setting of expectations and socialization of a new organizational members into an organization

[]19.13. P

Plan-Do-Check-Adjust (PDCA) cycle

PDCA (plan–do–check–act or plan–do–check–adjust) an iterative four-step management method used in business for the control and continuous improvement of processes and products. It is also known as the Deming circle/cycle/wheel, Shewhart cycle, control circle/cycle.[122]


It is helpful to conceptualize courseware as a combination of content and a player. The player software plays the courseware content as intended for blended ILT components, eLearning or mLearning. Player functionality contains more software as time goes on. Depending on the complexity of the game or simulation components, the player may also interact with courseware system models or human responsive models to provide dynamic interactions. The player contains the software code that provides the functionality to:

  1. {color:#000;}Provide the user interface for navigating through the content

  1. {color:#000;}Grade learner assessments
  1. {color:#000;}Use content APIs to get content from repositories for delivery (if used)
  1. {color:#000;}Provide user control for playing, pausing, and reversing audio and video content
  1. {color:#000;}Provide the learner or their screen reader accessibility content (when required)
  1. {color:#000;}Indicate decisions in branching simulation scenarios
  1. {color:#000;}Provide feedback content within the intended context
  1. {color:#000;}Retain session data so learners can return to the spot they left off
  1. {color:#000;}Report SCORM data (eLearning) to the LMS
  1. {color:#000;}Report xAPI data
  1. {color:#000;}Render style skins for look and feel
  1. {color:#000;}Receive and respond to mobile device sensor data (GPS, accelerometer, microphone, etc.)


The process by which a game designer and instructional/LX designer tests a new game for design flaws, pacing, and engagement before delivering it to the customer


An item that ideally satisfies a market’s want or need.[123] The output of a provisioning process for customer delivery. In project management, products are the formal definition of the project deliverables that make up or contribute to delivering the objectives of the project.[124] It may be a component of a higher level product or system.

product backlog

A backlog for the entire product. See backlog.


Performing a skill to its standard


Prototyping is constructing a small portion of a course or tool for the purpose of showing what the final product would be like. Ideally, this “piece” of the final product will be close to its final form to avoid rework. It is best to include an instance of any special content or functions of the final product as can be completed. The prototype can be used at any time, but is best used in presenting the proposed design of the asset to the customer stakeholders.


A term that describes how the provisioning process creates product by waiting on downstream customer demand. The opposite is push, which means the provisioning process creates product based on a forecasted schedule. Using a visual Kanban board, visual signals tell upstream process stations to produce for the downstream station/process step when it is empty and therefore needs more work items while the WIP limits communicate the specific quantity maximum of work items. As the downstream process steps (internal customers) consume their supply, they trigger an upstream process to replenish again.

[]19.14. Q

queue time

The time a work item spends in a line awaiting the next process step. Work in-process (WIP) waiting to be worked on. All queue time is delay, regardless of cause.

[]19.15. R


(1) Capability delivered to users, composed of multiple iterations. (2) The transition of an increment of potentially shippable product from the development team into use by customers. Releases typically happen when one or more iterations have resulted in the product having enough value to outweigh the cost to deploy it. The product is released to customer obligations. The release balances functionality, cost and quality requirements against date commitments. In training terms, a release is a course. Some training groups use Alpha Release and Beta Release, particularly for simulations and games. In publications, it is a set of manuals. (3) The designation by the identified release and control activity that configuration documentation/product information is approved by the appropriate authority and is subject to change management procedures. (4) The approved work authorization that authorizes the release of work products.


A singular documented physical and functional need that a particular design, product or process must be able to perform.[125] It is a statement that identifies a necessary attribute, capability, characteristic, or quality of a system for it to have value and utility to a customer, organization, internal user, or other stakeholder.[125] Sets of requirements are used as inputs into the design stages of [courseware] product development.[125] Requirements are also an important input into the verification process, since tests should trace back to specific requirements.[125]


Risk is the potential of gaining or losing something of value.[126] Values can be gained or lost when taking risk resulting from a given action or inaction, foreseen or unforeseen.[126] Risk can also be defined as the intentional interaction with uncertainty.[126] Uncertainty is a potential, unpredictable, unmeasurable and uncontrollable outcome; risk is a consequence of action taken in spite of uncertainty.[126] Risk is often gauged by both the probability of occurrence and the magnitude of its impact.

rolling wave planning

The process of project planning in waves as the project proceeds and later details become clearer.[127] Work to be done in the near term is based on high level assumptions; also, high level milestones are set.[127] As the project progresses, the risks, assumptions, and milestones originally identified become more defined and reliable.[127] When using Agile, rolling wave planning uses milestones weight with percent complete (MWPC) earned value technique (EVT). Assessing work-in-process can use control account manager (CAM) controlled planning of the milestone into “inchstones” with defined values. When using Agile, the work package period of performance should align with iteration boundaries (beginning of 1st iteration; end of last iteration). Milestone scope is the features planned for the milestone. Work package scope is the features planned for the release. Work packages and milestone budget is determined from feature scope.[29]

[]19.16. S


An imagined or projected sequence or development of events, courses of action, or situations intended to place a learner or performer in a context environment or circumstances as real as possible. We intentionally expand the definition to include stories, case studies, role plays, simulations and games. Scenarios usually include initial conditions and additional developments that result in a final situation. We also define a scenario to be a network of scenario nodes. We use nodes to describe the chunks of scenarios because the network of scenario nodes, or developments, can be linear or nonlinear in design. The instructional purpose is that the learning or skill gained through the scenario activity will be transferred to future real world situations.


Sharable Courseware Object Reference Model


An area on a device that displays content and functionality. In the past, screens were assumed to be monitor-sized. Today screen sizes vary depending on the device size.


Scrum is an iterative and incremental agile software development methodology for managing product development.[128] It defines a flexible, holistic product development strategy where a development team works as a unit to reach a common goal, challenges assumptions of the traditional, sequential approach to product development, and enables teams to self-organize by encouraging physical co-location or close online collaboration of all team members, as well as daily face-to-face communication among all team members and disciplines in the project.[128]


Arranging the key learning points into the most effective order for learning. Used in design to describe the order of presentation of learning objectives.


An economic activity where the buyer does not generally, except by exclusive contract, obtain exclusive ownership of the thing purchased. The benefits of such a service, if priced, are held to be self-evident in the buyer’s willingness to pay for it.[129]

simulation (sim)

A sim is any representation or imitation of reality.

  1. {color:#000;}Predictive simulation primarily involves data-model interaction.

  1. {color:#000;}Live simulation involves human-human interaction.
  1. {color:#000;}Virtual simulation involves human-computer interaction.
  1. {color:#000;}Training simulation is an instructional strategy used to teach problem solving, procedures, or operations by immersing learners in situations resembling reality. The learners actions can be analyzed, feedback about specific errors provided, and performance can be scored. They provide safe environments for users to practice real-world skills. They can be especially important in situations where real errors would be too dangerous or too expensive. Simulations provide a safe environment for performance error consequences, with low risk to learners and equipment. Sims can allow time compression. Organizations can use sims to gather experimental data.

size points

A means of conveying to our stakeholders and ourselves a rough order-of-magnitude sense of size and the risk landscape in which to understand that estimate.


Subject Matter Expert


See iteration. Sprint is a SCRUM method term. We use the more generic term iteration.

standard work

A lean tool that enables employees to work to proven work practices as the foundation from which we continuously improve. Typically standard work is a checklist. It helps make performance expectations clear. It can help new team members get up to speed more quickly. Standard work is not a replacement for skill and knowledge, its purpose is to enable skill and knowledge to be applied consistently and effectively.


A person, group or organization with an interest in a project.[130]


See user story.


A plan, like a blueprint, that outlines a learning experience. The storyboard is scoped at the lesson-level. For eLearning, it provides screen-level details. For blended instructor-led courseware, it provides the page-level presentation content, instructor guide content, rubrics for assessment, etc. Historically, storyboards have been intended to facilitate customer review of the detailed design at the level of the action-by-action flow at the screen-level/page-level. Lean-Agile proposes that the product increments do this better. If storyboards are still required, they are simply a state of the increment when text is entered without the remainder of the components built out yet. The storyboard includes text descriptions that describe virtual actions/system responses, actions of characters and their dialog, text headings, draft screen text and draft dialog script. It is used to more fully visualize the sequence of instruction, branch options, animation, interactions or simulation. The aesthetic quality of the storyboard may include text descriptions of the intended components and interaction sequences as placeholders. Historically, storyboards were applied to the entire courseware scope before development began. In Lean-Agile the storyboard, if used at all, is only built for the LO under one-piece development and never more than the LOs in that iteration’s scope.

Subject Matter Expert (SME)

A person who can perform a job or a selected group of tasks to standards. Their experience and knowledge of the job designates them as a technical expert. They must know what is critical to the performance of the task and what is nice-to-know. They must have recent job experience otherwise, their knowledge of the task may be outdated by new procedures or equipment. They act as a resource to the course designer.

successive approximation method (SAM)

An agile method of training development proposed by Michael Allen in an ASTD published book, Leaving ADDIE for SAM, published in 2012.

[]19.17. T

Theory of Constraints

The theory of constraints (TOC) is a management paradigm that views any manageable system as being limited in achieving more of its goals by a very small number of constraints.[131] There is always at least one constraint, and TOC uses a focusing process to identify the constraint and restructure the rest of the organization around it.[131] TOC adopts the common idiom “a chain is no stronger than its weakest link.”[131] This means that processes, organizations, etc., are vulnerable because the weakest person or part can always damage or break them or at least adversely affect the outcome.[131]


Throughput is the movement of inputs and outputs through a production process. Without access to and assurance of a supply of inputs, a successful business enterprise would not be possible.[132] In the theory of constraints, throughput is the rate at which a system achieves its goal.[132] A measure of the volume of work items passing through a system from input to output over time. The rate of production; the rate at which something can be processed. Also called velocity, inventory turn-over or rate of flow.


Timeboxing is a planning technique common in planning Scrum projects (typically for software development), where the schedule is divided into a number of timeboxes.[133] In time management, timeboxing allocates a fixed time period, called a time box, to each planned activity.[133] It often involves having deliverables and deadlines.[133] Without timeboxing, projects usually work to a fixed scope.[133] Timeboxes are used as a form of risk management.[133]


Theory of Constraints


In software, a toolchain is the set of programming tools that is used to perform a complex software development task or to create a software product, which is typically another computer program or a set of related programs. In general, the tools forming a toolchain are executed consecutively so the output or resulting environment state of each tool becomes the input or starting environment for the next one, but the term is also used when referring to a set of related tools that are not necessarily executed consecutively.[134]

[]19.18. U

user story or story

A user story is something a user wants (part of a feature). It is a work item where the effort is can be finished during a single iteration (2-4 wks). The origins of this term are from software Lean-Agile projects, so software people may already know the term. Engineering groups also call these requirements. One prominent Agile consultant, Mountain Goat Software, recommends a format for user stories as follows: “As a [type of user] I [want/can/am able to/need to] so that [some reason].”

[]19.19. V

visible management

A Lean principle. Making visible to workers, managers and even customers the work and its execution in ways that convey the daily situation at a glance. Invisible work cannot be improved. Kanban cards are examples of Visible Management, as are tool shadow boards and storyboards. Visual management provides data. Showing your knowledge work processes on a Kanban board helps make the process and the status of the work in process visible to the team.

[]19.20. W


Anything (time, costs, work) that adds no value in the eyes of the customer. Lean states there are seven types of waste: (1) overprocessing, (2) transportation, (3) motion, (4) inventory, (5) waiting time, (6) defects, (7) overproduction.

work in-process (WIP)

A work item that is partially completed (an incomplete state) at a specified time. WIP includes work item inventory between process steps, in queues, in in-trays, out-trays, in server folders, on trucks or in stores as materials or finished goods. WIP can also be expressed financially as the value of resources that have been invested toward its provisioning at the end of an accounting period. WIP is increased by long setup times, rework, queue times, variation in supply and demand and complexity. A goal of Lean is to reduce WIP to shrink lead time. In Little’s Law (Lead Time = Amount of Work-in-Process (WIP) / Average Completion Rate), if you reduce the numerator, lead time shrinks, even with no improvement in completion rate. Limit the amount of work allowed into the process. This takes intellectual capital only. Increasing completion rates, with people or machines, takes capital. Training professionals are just beginning to think of their work items as work in-process.

work flow

Work flow is the progressive unimpeded adding of value to an item or service as it moves through the organization from order to delivery.

work in-process (WIP)

A work item that is partially completed (an incomplete state) at a specified time. WIP includes work item inventory between process steps, in queues, in in-trays, out-trays, in server folders, on trucks or in stores as materials or finished goods. WIP can also be expressed financially as the value of resources that have been invested toward its provisioning at the end of an accounting period. WIP is increased by long setup times, rework, queue times, variation in supply and demand and complexity. A goal of Lean is to reduce WIP to shrink lead time. In Little’s Law (Lead Time = Amount of Work-in-Process (WIP) / Average Completion Rate), if you reduce the numerator, lead time shrinks, even with no improvement in completion rate. Limit the amount of work allowed into the process. This takes intellectual capital only. Increasing completion rates, with people or machines, takes capital. Training professionals are just beginning to think of their work items as work in-process.

work item (general)

A chunk or unit of work that is similarly-sized to help create constant flow through a process. In service work the ability to see the work moving through the process has to be explicitly added, so we use visual management.

work item (training)

In training, granularity levels include:

  1. {color:#000;}Curriculum

  1. {color:#000;}Course, courseware, game application
  1. {color:#000;}modules, game levels
  1. {color:#000;}lessons, game missions
  1. {color:#000;}job duties, competencies, terminal LOs
  1. {color:#000;}job tasks, enabling LOs
  1. {color:#000;}teaching points or key activity/scenario nodes (situation, scene, event, branching node, demonstration, practice iteration) in linear pages or a nonlinear node network
  1. {color:#000;}test item, graphic elements, animation, video, audio, action, step, dialog node, interaction (not pages)
  1. {color:#000;}work tasks

work item (immersive virtual environments)

In immersive virtual environments granularity levels include:

  1. {color:#000;}virtual environment system

  1. {color:#000;}virtual area/scene structure, virtual subsystem behaviors
  1. {color:#000;}virtual object structure, virtual object behaviors
  1. {color:#000;}virtual component structure, virtual component state transitions
  1. {color:#000;}work tasks

work item (agile software)

In agile software that supports training, granularity levels include:

  1. {color:#000;}software application

  1. {color:#000;}epics
  1. {color:#000;}features
  1. {color:#000;}user stories
  1. {color:#000;}work tasks

work item (technical manuals)

In technical manuals granularity levels include:

  1. {color:#000;}publication sets/collections

  1. {color:#000;}manuals
  1. {color:#000;}chapters
  1. {color:#000;}sections
  1. {color:#000;}topics/data modules/work packages
  1. {color:#000;}interaction/2D or 3D animation/video/audio/graphic elements
  1. {color:#000;}work tasks

|={color:#000;background:rgba(0, 0, 0, 0);}. Note|<{color:#000;background:rgba(0, 0, 0, 0);}. Use the semantic names for work items (what they are) rather than media-based names (their output format in a particular media for a particular device). For example, a chapter may be 25 pages in paper or PDF, but 50 pages on an e-reader, or one long scrolling page in HTML5.|

work product

A process artifact or output. In learning experience development, this may include courseware, storyboards, course design documents, instructor guide, student guide, learner assessments, etc. It may be from interim process steps. It does not have to be the entire final product or course.

[]20. Notes

1. See the book Lean for Systems Engineering, by Bohdan W. Oppenheim, 2011 for more about how Lean has been adopted within the Systems Engineering discipline.

2. Quoted with permission from Mary Poppendieck from the introduction of Lean Software Development: An Agile Toolkit, by Mary Poppendieck and Tom Poppendieck, 2003

3. Reproduced with permission from Firas Glaiel, Allen Moulton and Stuart Madnick from their paper, Agile Project Dynamics: A System Dynamics Investigation of Agile Software Development Methods, by Firas S. Glaiel, Allen Moulton and Stuart E. Madnick, 2013.

4. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Printing_press under a Creative Commons Attribution-ShareAlike 3.0 license.

5. We recommend Atul Gawande’s excellent book, The Checklist Manifesto: How to Get Things Right

6. Reproduced from Wikipedia article “Waterfall model,” https://en.wikipedia.org/w/index.php?title=Waterfall_model&oldid=688385747 (accessed December 14, 2015) under a Creative Commons Attribution-ShareAlike 3.0 license.

7. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Taiichi_Ohno under a Creative Commons Attribution-ShareAlike 3.0 license.

8. Wikipedia contributors, “Lean manufacturing,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Lean_manufacturing&oldid=698653017

9. For more details, see The Machine That Changed the World by James P. Womack, Daniel Roos and Daniel T. Jones, 1990.

10. Wikipedia contributors, “Toyota Production System,” Wikipedia, The Free Encyclopedia,https://en.wikipedia.org/wiki/Toyota_Production_System

11. See Lean Thinking, by James P. Womack and Daniel T. Jones, 1996.

12. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Lean_thinking under a Creative Commons Attribution-ShareAlike 3.0 license.

13. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/PDCA under a Creative Commons Attribution-ShareAlike 3.0 license.

14. For more detail about how the Japanese used quality before the American automakers, see The Reckoning, by David Halberstam, 1986

15. http://services.leankanban.com/brief-history-kanban-knowledge-work

16. see http://www.clarkaldrichdesigns.com/2007/01/big-skills.html

17. Reproduced with permission from Clark Aldrich, http://www.clarkaldrichdesigns.com/2007/01/big-skills.html

18. Some credit Michael Molenda for the creation of the commonly used ADDIE model. If interested in the history of ADDIE, read In Search of the Elusive ADDIE Model, by Michael Molenda of Indiana University, published in Performance Improvement, May/June 2003 by ISPI.org

19. Wikipedia contributors, “ADDIE Model,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=ADDIE_Model&oldid=701779637

20. Wikipedia contributors, “Winston W. Royce,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Winston_W._Royce&oldid=684180641

21. Reproduced with permission from Aaron Erickson, author of Everybody’s Doing Agile—​Why Can’t We?, 2011, http://www.informit.com/articles/article.aspx?p=1743015

22. Reproduced with permission from Linda Rouleau, REVISITING PERMANENTLY-FAILING ORGANIZATIONS: A PRACTICE PERSPECTIVE, by Linda Rouleau, Stéphanie Gagnon et Charlotte Cloutier, 2008 http://expertise.hec.ca/geps/wp-content/uploads/2013/09/GePS-08-01.pdf

23. See later section on Continuous Improvement – Kaizen

24. EVM description reproduced from the U.S. Goverment site http://www.acq.osd.mil/evm/faqs.shtml#q11. As such it is not entitled to domestic copyright protection under U.S. law. See https://acc.dau.mil/CommunityBrowser.aspx?id=17609 for more information about EVM if desired.

25. Reproduced from the U.S. Goverment site http://www.gao.gov/htext/d08896.html and developed by the U.S. Government Accountability Office (GAO). As such it is not entitled to domestic copyright protection under U.S. law.

26. PARCA is part of the U.S. Goverment. It is a Directorate of the Office of the Assistant Secretary of Defense for Acquisition, http://www.acq.osd.mil/parca/about.shtml. As such its works are not entitled to domestic copyright protection under U.S. law.

27. PARCA’s meeting was reported at http://www.acq.osd.mil/evm/resources/EVM-Agile%20Meeting.html

28. Don Johnson, Office of the Secretary of Defense, http://www.acq.osd.mil/evm/resources/AgileMtgSlides_andAgenda/1%20Johnson_PARCA_EVM_Agile.pdf. As an employee of the US Government in official duties, his statements are not entitled to domestic copyright protection under U.S. law.

29. Jim Duffy, Raytheon, http://www.acq.osd.mil/evm/resources/AgileMtgSlides_andAgenda/EVMS%20for%20Agile%20Projects%20PARCA%202015%2002%2019.pdf

30. Reproduced with permission from Thomas Blackburn, Attain Group, http://www.acq.osd.mil/evm/resources/AgileMtgSlides_andAgenda/9%20Blackburn_Key%20Performance%20Measures%20in%20a%20Lean%20Agile%20ProgramOriginal.pdf

31. Robert Gates, the United States Secretary of Defense, 2008, Agile Methods: Selected DoD Management and Acquisition Concerns, Carnegie Mellon University, October 2011, p. ix. As an employee of the US Government in official duties, his statements are not entitled to domestic copyright protection under U.S. law.

32. Reproduced with permission from JoAnn Hackos, PhD.

33. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/System under a Creative Commons Attribution-ShareAlike 3.0 license.

34. Reproduced from https://en.wikipedia.org/wiki/Complex_systems under a Creative Commons Attribution-ShareAlike 3.0 license.

35. Reproduced from https://en.wikipedia.org/wiki/Complex_system under a Creative Commons Attribution-ShareAlike 3.0 license.

36. Reproduced from Wikipedia article “Complex adaptive system,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/wiki/Complex_adaptive_system (accessed January 14, 2016) under a Creative Commons Attribution-ShareAlike 3.0 license

37. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Causality under a Creative Commons Attribution-ShareAlike 3.0 license.

38. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Systems_analysis under a Creative Commons Attribution-ShareAlike 3.0 license.

39. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Complex_system under a Creative Commons Attribution-ShareAlike 3.0 license.

40. Reproduced from Scholarpedia article http://www.scholarpedia.org/article/Complexity under a Creative Commons Attribution-ShareAlike 3.0 license.

41. https://en.wikipedia.org/wiki/System_accident

42. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Distributed_artificial_intelligence, under a Creative Commons Attribution-ShareAlike 3.0 license.

43. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Complex_adaptive_system, under a Creative Commons Attribution-ShareAlike 3.0 license.

44. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Computational_economics under a Creative Commons Attribution-ShareAlike 3.0 license.

45. Reproduced with permission from Jurgen Appelo from http://www.slideshare.net/jurgenappelo/complexity-thinking/67-Complexity_theory_predicts_that_we. See his book Management 3.0: Leading Agile Developers, Developing Agile Leaders (Addison-Wesley Signature Series), 2011, by Jurgen Appelo for more about how Agile helps complexity.

46. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Emergence under a Creative Commons Attribution-ShareAlike 3.0 license.

47. Reproduced with permission from Jurgen Appelo from http://www.slideshare.net/jurgenappelo/complexity-thinking/94-A_key_point_of_complexity. See his book Management 3.0: Leading Agile Developers, Developing Agile Leaders (Addison-Wesley Signature Series), 2011, by Jurgen Appelo for more about how Agile helps complexity.

48. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Cynefin_Framework under a Creative Commons Attribution-ShareAlike 3.0 license.

49. Emerson published his works in the U.S. before 1923. As such copyright has expired into the public domain.

50. Reproduced by permission stated on http://agilemanifesto.org/, “may be freely copied in any form”, 2001 copyright belongs to the 17 authors listed on that site. We have only added courseware in brackets to indicate how the same idea applies to training.

51. See the later section about Using Queuing Theory to Improve Kanban Traffic Jams.

52. Reproduced with permission from The Lean Enterprise Institute, Inc. (LEI), © Copyright 2016, Lean Enterprise Institute, Inc., Cambridge, MA, lean.org. All rights reserved., See http://www.lean.org

53. We recommend reading A Leader’s Framework for Decision Making, by David J. Snowden and Mary E. Boone, Harvard Business Review, November 2007, for a better description of how complexity impacts decision making.

54. For two new waste types, see Lean Product Development: Making waste transparent, by Christoph Bauch Ph.D. Massachusetts Institute of Technology 2004

55. To read Jamie’s blog entry, see Standard work is not a replacement for skill and knowledge, by Jamie Flinchbaugh on May 29, 2012. http://jamieflinchbaugh.com/2012/05/standard-work-is-not-a-replacement-for-skill-and-knowledge/.

56. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Kaizen under a Creative Commons Attribution-ShareAlike 3.0 license.

57. General George S. Patton, United States Army. As an employee of the US Government in official duties, his statements are not entitled to domestic copyright protection under U.S. law.

58. General Dwight D. Eisenhower, United States Army. As an employee of the US Government in official duties, his statements are not entitled to domestic copyright protection under U.S. law.

59. The Learning Environment Modeling Language, by the University of Central Oklahoma. For more information, see their site at http://uco.edu/lem

60. Reproduced with permission from Damien Newman, http://revisionlab.wordpress.com/that-squiggle-of-the-design-process/

61. Reproduced with the permission of A List Apart (alistapart.com) and Cennydd Bowles from Getting Real About Agile Design, by Cennydd Bowles, 2008, http://alistapart.com/article/gettingrealaboutagiledesign

62. See Cathy Moore’s blog for her Action Mapping model and her training on how to use it. http://blog.cathy-moore.com/action-mapping-a-visual-approach-to-training-design/

63. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Requirement under a Creative Commons Attribution-ShareAlike 3.0 license.

64. This restaurant example is not our original idea, but we don’t remember who first mentioned it because we came across it early. The idea created a mental picture that stuck. We’re not sure who to attribute. If you know who said this first, let us know and we’ll attribute it appropriately.

65. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Little%27s_law under a Creative Commons Attribution-ShareAlike 3.0 license.

66. Reproduced from Wikipedia article “Cynefin Framework,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Cynefin_Framework&oldid=698757685 (accessed January 12, 2016) under a Creative Commons Attribution-ShareAlike 3.0 license.

67. Read David’s history of his framework at http://cognitive-edge.com/blog/part-two-origins-of-cynefin/

68. David Snowden’s 2014 image of his Cynefin Framework is reproduced from https://en.wikipedia.org/wiki/Cynefin under the Creative Commons Attribution-Share Alike 3.0 Unported license.

69. Reproduced with permission from Jurgen Appelo, http://www.slideshare.net/jurgenappelo/complexity-thinking.

70. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Capability_Maturity_Model_Integration under a Creative Commons Attribution-ShareAlike 3.0 license.

71. Reproduced with permission from Jurgen Appelo from http://noop.nl/2011/11/networked-kanban.html.

72. Reproduced with permission from Ilan Goldstein, Axis Agile, Copyright 2013. http://www.amazon.com/Scrum-Shortcuts-without-Cutting-Corners/dp/0321822366

73. Reproduced with permission from Tim Nolan, Agile Government, Collin County Texas, http://www.slideshare.net/TimNolan3/agile-government

74. PLANNING POKER ® is a registered trademark of Mountain Goat Software, LLC

75. Reproduced with permission from Mountain Goat Software.

76. Reproduced with permission from Dominik Maximini, Estimating relative sizes, http://scrumorakel.de/blog/index.php?/archives/48-Estimating-relative-sizes-e.g.-story-points.html

77. Reproduced from https://gist.github.com/zmwangx/9987772 under a WTFPL license (http://www.wtfpl.net/).

78. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Scrum_(software_development) under a Creative Commons Attribution-ShareAlike 3.0 license.

79. See https://en.wikipedia.org/wiki/IBM_Generalized_Markup_Language

80. See http://www.w3.org/MarkUp/SGML/

81. See http://www.w3.org/XML

82. See http://www.json.org/

83. See http://html5.org

84. See http://www.w3.org/Style/CSS/Overview.en.html

85. See http://www.w3.org/Style/XSL/WhatIsXSL.html

86. See http://daringfireball.net/projects/markdown

87. See http://asciidoc.org

88. See http://pandoc.org

89. See http://fletcherpenney.net/multimarkdown

90. See http://www.latex-project.org/

91. See http://xmlgraphics.apache.org/fop/

92. See https://acrobat.adobe.com/us/en/products/about-adobe-pdf.html

93. See http://idpf.org/epub/301

94. See http://www.w3.org/TR/xslt20/

95. See https://en.wikipedia.org/wiki/JavaScript

96. See http://dita.xml.org/

97. See https://git-scm.com/

98. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Agile_software_development under a Creative Commons Attribution-ShareAlike 3.0 license.

99. Reproduced from Wikipedia article “Black box,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Black_box&oldid=695218886 (accessed December 14, 2015) under a Creative Commons Attribution-ShareAlike 3.0 license.

100. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Burn_down_chart under a Creative Commons Attribution-ShareAlike 3.0 license.

101. See more about the Pomodoro technique at http://en.wikipedia.org/wiki/Pomodoro_Technique

102. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Continual_improvement_process under a Creative Commons Attribution-ShareAlike 3.0 license.

103. Reproduced from Wikipedia article “Coupling (computer programming),” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Coupling_(computer_programming)&oldid=689840780 (accessed December 14, 2015)under a Creative Commons Attribution-ShareAlike 3.0 license.

104. Reproduced from the U.S. Goverment site http://www.adlnet.gov/help/Pages/Terms.aspx. As such it is not entitled to domestic copyright protection under U.S. law.

105. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Cynefin_Framework under a Creative Commons Attribution-ShareAlike 3.0 license.

106. Reproduced from U.S. Government MIL-HDBK-29612-2A, As such it is not entitled to domestic copyright protection under U.S. law.

107. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Corrective_feedback under a Creative Commons Attribution-ShareAlike 3.0 license.

108. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Video_game under a Creative Commons Attribution-ShareAlike 3.0 license.

109. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Interactive_fiction under a Creative Commons Attribution-ShareAlike 3.0 license.

110. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Inspection under a Creative Commons Attribution-ShareAlike 3.0 license.

111. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Interaction under a Creative Commons Attribution-ShareAlike 3.0 license.

112. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Interaction_design under a Creative Commons Attribution-ShareAlike 3.0 license.

113. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Iteration under a Creative Commons Attribution-ShareAlike 3.0 license.

114. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Scrum_(software_development)#Sprint under a Creative Commons Attribution-ShareAlike 3.0 license.

115. Reproduced from U.S. Government MIL-HDBK-29612-4A. As such it is not entitled to domestic copyright protection under U.S. law.

116. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Kaizen under a Creative Commons Attribution-ShareAlike 3.0 license.

117. For more information about Key Learning Points, see the U.K. Ministry of Defence manual Defence systems approach to training (JSP 822), https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/28531/jsp822part2.pdf

118. Wikipedia contributors, “Donald Kirkpatrick,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Donald_Kirkpatrick&oldid=665576728 (accessed January 5, 2016).

119. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Learning_curve under a Creative Commons Attribution-ShareAlike 3.0 license.

120. Reproduced from The Glossary of Education Reform entry http://edglossary.org/learning-experience/under a Creative Commons Attribution-ShareAlike 4.0 license.

121. See also Connie Malamed’s blog entry and rationale about using LX design instead of instructional design at http://theelearningcoach.com/elearning_design/isd/new-name-for-id/.

122. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/PDCA under a Creative Commons Attribution-ShareAlike 3.0 license.

123. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Product under a Creative Commons Attribution-ShareAlike 3.0 license.

124. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Product_(business) under a Creative Commons Attribution-ShareAlike 3.0 license.

125. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Requirement under a Creative Commons Attribution-ShareAlike 3.0 license.

126. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Risk under a Creative Commons Attribution-ShareAlike 3.0 license.

127. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Rolling_Wave_planning under a Creative Commons Attribution-ShareAlike 3.0 license.

128. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Scrum_(software_development) under a Creative Commons Attribution-ShareAlike 3.0 license.

129. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Service_(economics) under a Creative Commons Attribution-ShareAlike 3.0 license.

130. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Stakeholder under a Creative Commons Attribution-ShareAlike 3.0 license.

131. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Theory_of_constraints under a Creative Commons Attribution-ShareAlike 3.0 license.

132. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Throughput_(business) under a Creative Commons Attribution-ShareAlike 3.0 license.

133. Reproduced from Wikipedia article https://en.wikipedia.org/wiki/Timeboxing under a Creative Commons Attribution-ShareAlike 3.0 license.

134. Reproduced from Wikipedia article, “Toolchain,” Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Toolchain&oldid=689948879 (accessed December 14, 2015) under a Creative Commons Attribution-ShareAlike 3.0 license.

Agile Courseware - How Teams Can Make More Learning Experiences in Less Time

Learn how others are getting courseware developed more quickly. Raytheon shares their journey and experiences applying Lean and Agile principles from other disciplines into an improved way of developing training courseware. Agile Courseware is aimed at two audiences: executives that buy courseware and teams that produce courseware. This book can help you buy or make courseware more quickly too. Like an extended session at an industry conference, Agile Courseware shares Raytheon’s lessons of success to help you calibrate your expectations. They also warn of common struggles while translating Agile and Lean into training product development to ease your journey. It tells how Agile changes courseware development and offers a common vocabulary you can use. Courseware Buyers will find out: * What to do differently to accommodate the Agile courseware development * How Agile changes your courseware buying processes * How the Agile Courseware approach impacts buyer’s risk * Expectations for buyer’s staff that support your transition * How the Agile Courseware approach is different from traditional ADDIE Courseware Producers will find out: * Executive level considerations for CLOs and VPs considering new Agile Courseware initiatives * How teams can execute projects quickly enough so that timely interventions address the current business need * How Agile Courseware impacts ADDIE and the changes for instructional designers. * How to mitigate risk and how to troubleshoot problems * How managers and teams can visualize progress to stay informed * Shows the new and the experienced with Agile how to adapt it to learning and development. So get started today with quicker courseware development. Pick up this book now.

  • Author: Kevin Lanham
  • Published: 2016-03-18 19:20:47
  • Words: 86259
Agile Courseware - How Teams Can Make More Learning Experiences in Less Time Agile Courseware - How Teams Can Make More Learning Experiences in Less Time