O'Reilly logo
live online training icon Live Online training

Getting started with machine learning

An introduction to developing production-ready solutions using systems thinking principles

Topic: Data
Kishau Rogers

The future of software is being driven by intelligent applications, and 81% of IT leaders are investing or plan to invest in artificial technology solutions. Given the large amounts of data and information available, more companies are pivoting from preprogrammed software applications to intelligent applications that leverage machine learning algorithms. However, although machine learning is a powerful tool for accelerating the rate of learning, it can also lead to unintended and dangerous consequences that are directly linked to the people and data that train the machines to learn. Avoid these issues by learning to better evaluate, analyze, and understand your systems.

Join expert Kishau Rogers for a hands-on introduction to machine learning. You'll explore scalable project workflows and common pitfalls and walk through the steps for developing and deploying production-ready machine learning solutions. Throughout the course, Kishau incorporates systems thinking principles as a foundation for improving algorithmic performance, providing a deeper understanding of the assumptions and elements (systems) that will influence (train) your models. This hybrid approach integrating machine learning with systems thinking improves a project's effectiveness and ultimately supports the goal of developing machine learning solutions that make a positive impact in the areas that matter.

What you'll learn-and how you can apply it

By the end of this live online course, you’ll understand:

  • Machine learning concepts and where ML fits in within the artificial intelligence ecosystem
  • Systems thinking concepts and why systems thinking is important in machine learning projects
  • The workflow for developing machine learning tools
  • Machine learning types and common algorithms by project goal
  • Common pitfalls for building and validating models
  • How to use systems thinking tools such as system archetypes, the iceberg model, and causal loop diagrams for defining system patterns to avoid common pitfalls of deploying production-ready machine learning tools
  • How to develop a roadmap for your machine learning project

And you’ll be able to:

  • Clarify the question to be answered with machine learning
  • Identify adequate datasets
  • Prepare and clean datasets
  • Validate models and evaluate for overfitting and bias
  • Deploy models in production
  • Monitor and adjust to changes with anomaly detection
  • Define a system’s archetype, the common and recurring patterns of a system
  • Recognize features by cycles presented in feedback and causal loop diagrams
  • Transform your idea discovery and problem-solving processes by identifying a system’s high leverage point

This training course is for you because...

  • You're a software developer interested in developing production-ready machine learning solutions and need to understand the typical workflow and pitfalls for machine learning projects.
  • You're the technical leader or owner of a software product seeking to implement a workflow for integrating machine learning solutions into your product(s) and software development operations.


  • Prior programming experience, ideally in Python or R
  • A working knowledge of data analysis (e.g., handling and analyzing data files—cleaning, preparing, visualizing, etc.)

Recommended preparation:

The State of Artificial Intelligence (video)

Can We Build AI Without Losing Control of It? (video)

"Attacking Machine Learning with Adversarial Examples” (article)

"Infrastructure for Deep Learning" (article)

Peter Senge's Introduction to Systems Thinking (video)

The System Orientation: From Curiosity to Courage” (article)

"What Is Systems Thinking? The Evolution of the System for Thinking About Systems" (article)

How the Game Is Evolving” (chapter 1 in Systems Thinking)

Less Coding, More Thinking” (podcast episode)

About your instructor

  • Kishau Rogers is the founder of Websmith Inc., which has been providing technology solutions to organizations across the globe since 2004. Under Kishau's leadership, Websmith Inc. has contributed to the launch of a number of successful startups, including health enterprise solution TimeStudy, innovative geolocation platform PeerLoc, and technology consulting agency bigThinking. A serial tech entrepreneur, Kishau has been developing impactful, innovative solutions to complex problems and business challenges in the computer science and information technology industry for over 23 years. At the core of Kishau’s work is an intention to make a real, positive difference in the world. She serves her community as an advisor, board member, and thought leader for social impact technology projects including Think of Us, a technology startup on a mission to improve the outcomes for youth in care; the US White House Hackathon for Foster Care; the WAAW Foundation; and SheHacks, a software engineering intensive program for women and girls in Africa offered by the WAAW Foundation. Kishau has been featured in national publications including the Wall Street Journal, Entrepreneur, NFIB, and JavaWorld and is a recipient of the NAWBO Wells Fargo STEM award, the Lyn McDermid Community Impact Award, and the MBL Entrepreneur of the Year Award.


The timeframes are only estimates and may vary according to how the class is progressing

Working definitions (5 minutes)

  • Lecture: A working definition of AI and its disciplines as defined by goals—planning, knowledge representation, perception, robotics, natural language processing (NLP), and machine learning; artificially intelligent systems versus preprogrammed systems; a working definition of systems thinking and a comparison with other thinking disciplines; how systems thinking can support the machine learning project workflow

Machine learning (ML) orientation (10 minutes)

  • Lecture: Knowing when to use machine learning (and when to use the term); common misunderstandings regarding machine learning versus artificial intelligence; real-world applications of machine learning; the characteristics of machine learning applications

Classifying applications as ML projects versus preprogrammed or other technology projects (20 minutes)

  • Lecture: How to form a question that can be answered with data; the types of ML systems and the purpose of each; choosing and defining the boundaries of a whole system at a level suitable for the particular purpose under consideration
  • Hands-on exercise: Clarify your purpose and question to determine the type of model that you’re building

Data acquisition (20 minutes)

  • Lecture: Finding appropriate data assets to answer the question; identifying and evaluating data assets; using the iceberg model to evaluate data assets, events and transactions, rules, policies, limiting factors, delays, and historical context
  • Hands-on exercise: Complete an iceberg model diagram for your data assets

Break (10 minutes)

Data preparation and cleaning (20 minutes)

  • Lecture: Avoiding low-quality data (i.e., inconsistent, duplicate, corrupt datasets); eliminating the unnecessary and identifying the appropriate features; building the ability to see complex relationships; articulating reality and defining boundaries (with features)
  • Hands-on exercise: Implement feature selection via feedback loops

Model building (20 minutes)

  • Lecture: Recognizing and avoiding under- and overfitting (i.e., building a model that fails to learn); using a supervised prediction model use case to understand the statistics behind modeling; a holistic review of patterns and trends over time as a part of the natural dynamics of system
  • Hands-on exercise: Identify training sets and test sets (on paper)

Evaluation and validation (20 minutes)

  • Lecture: Identifying biased datasets before training; identifying precision, sensitivity, specificity, and accuracy
  • Hands-on exercise: Evaluate performance and complete a matrix of true/false positives/negatives based on the perspective of the positive class

Break (10 minutes)

Deploying (20 minutes)

  • Lecture: Building models that add real-world value; exploratory versus production prediction engines—determining how this ML system fits in the larger system and the appropriate point of intervention; identifying cause and effect using an expansion of the feedback loop created during data preparation
  • Hands-on exercise: Use causal loop diagrams to determine the point of highest leverage (i.e., the point in the system where the ML solution offers the highest value)

Monitoring (15 minutes)

  • Lecture: Monitoring models for adaptability; how to use anomaly detection algorithms to flag unexpected, weird changes in behaviors over time; how to determine the impact of changes of time; managing interactions rather than actions; common system archetypes

Wrap-up and Q&A (10 minutes)