book

Unifying Business, Data, and Code

by Ron Itelman, Juan Cruz Viotti

January 2024

Intermediate to advanced

356 pages

9h 53m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

What You Can’t See Can Kill You, and the Same Is True for DataHidden Threats to Organizations: A Modern ParallelYour AI Is Only as Good as Your DataAligning Problem-Solving Strategies, Data, and AIA New Paradigm to Optimize Data Management and Business Strategy for the Age of AIThe Origin Story of UnifyingOrchestrating Alignment at Organizational ScaleConventions Used in This BookO’Reilly Online LearningHow to Contact UsAcknowledgments
Your Quest for Data-Driven Breakthroughs Begins There Are Usually Multiple, Conflicting North StarsThe Good, the Bad, and the Ugly of Data ProblemsThe Problem with ProblemsUnifying Concepts: The Key to InnovationWhat a Unifying Data Strategy Will Do for AgileDefining Being AgileAgile TheaterAgile, Waterfall, and UnifyingDefining a Unifying Data Strategy ApproachUnderstanding the Phrase Being Data DrivenTo Be Data Driven, Be Data CentricBottlenecks Preventing Teams from Being Data DrivenThis Book’s Project: Intelligence.AI Coffee BeansSummary
Introducing JSONA Simple JSON ExampleJSON Viewing and Authoring ToolsOverview of JSON GrammarBooleansNumbersStringsArraysObjectsNullLearning MoreMinificationAlternative RepresentationsCreating a JSON DocumentA Product EntryA Store OrderSummary
Data Transformations Require Data ChampionsThe Rise of the Data Product ManagerAlignment Is a Journey, Not a DestinationEvaluating Alignment from a Holistic PerspectiveThe Goal Isn’t Alignment, It’s Effective AlignmentStrategies for Setting Up Teams for SuccessIncorporating a Product Management MindsetDefining Data Users’ NeedsDefining Product FeaturesDefining and Measuring SuccessUnifying Versus AligningSummary
Packaging and Products: An Example Using CoffeeThe Four Facets of a Data ProductGetting Started with Concept-First DesignA Blueprint for UnifyingMapping the Conceptual Terrain: Assessing ConceptsFacilitating Assessments of Conceptual Alignment Across Technical and Nontechnical TeamsSmooth Is Slow, Slow Is FastSummary
What Is JSON Schema?What Is a Schema?The Building Blocks of JSON SchemaVocabularies and DialectsMeta-Schemas: Schemas That Describe Other SchemasUnderstanding JSON SchemasStep 1: Determining the Schema Dialect: The $schema KeywordStep 2: Determining the Schema VocabulariesStep 3: Understanding Schema VocabulariesStep 4: Understanding Schema KeywordsJSON Schema as a Recursive Data StructureReferencing SchemasWhat does duplication look like?Local referencingRemote referencingYour First JSON Schema ProjectWriting a Schema: Step by StepGenerating a Web FormSummary
Enemies of Alignment: Ambiguity and AssumptionsAmbiguity: The Culprit in the Illusion of CommunicationAssumptions: Ambiguity’s Best FriendDefining Success: Symmetry Between Concepts and JSON Schema Equals Minimal AmbiguityIlluminating Misalignment with a Concept CompassStep 1: Harmonizing the WhatStep 2: Harmonizing the WayStep 3: Harmonizing the HowHarmonized Concepts Validating Concepts: Belief Scoring and HypotheticalsCounterfactualsBelief ScoringSummary
An Introduction to Thinking in NetworksExample of Thinking in Networks: Athletes Versus ArtistsGraphs: The Visual Language of NetworksNetworks of Entities: Knowledge GraphsA Simple Knowledge GraphChallenges with Knowledge GraphsAligning Knowledge for the 99%Fundamentals of CLEAN Data GovernanceCollaborationKnowledgeBusiness LogicActivityCLEAN Data Governance in PracticeThe Four Facets of Data Products and CLEANThe Four Horsemen of Data DeathIgnoranceSiloed IncentivesShortsightednessIndecisivenessThe Power of Design in Collaborative NetworksSummary
Validating the Structure of DataUsing an Online ValidatorValidation ExampleJSON Schema as a Constraints LanguageBoolean SchemasHeterogeneous Data StructuresThe format KeywordUsing Annotations to Define MeaningAnnotation Extraction ExampleA Simple Use Case: DeprecationsRuntime ExtractionStandard Output FormatsRevisiting the format KeywordUsing an Online ValidatorThinking in SchemasSummary
How Ambiguity, Knowledge Gaps, and Blind Spots Influence Decisions and Progress Toward GoalsWhich Is Bigger: Greenland or the US?Mapping Pathways of Processes and ProgressMeasuring Progress Toward GoalsDefining Decisions and Steps with Process MapsHow Process Maps Reveal AmbiguityVisualizing and Removing Ambiguity in ProcessesEnriching Process Maps with AnnotationsProcess Maps Reveal Innovation OpportunitiesSummary

An Introduction to Knowledge FrameworksKnowledge Experiences and PathwaysA Tool for Designing Knowledge ExperiencesFrom Structured Knowledge to Computational KnowledgeSuccess SpectrumsMapping Progress and ValueVisualizing and Adding “Next Best States” Removing Blind SpotsEmbracing Multiperspective Design and Road MapsDefining KPIs for Success Measures and Metrics (Assessments)Using Demons and Magical Thinking for InnovationFaster HorsesImagining Magical PossibilitiesProblem Landscapes: Quantifying Pain Points Threatening ValueNudges: The Right Information at the Right TimeA Real-Life Problem Landscape and Demon Example That Led to a Unified Data Product ModelUnderstanding the Problem LandscapeThe Staggering ImpactA Meeting of Minds and the Birth of a SolutionBeyond Data Products: Data Product ManagementThe Circular Nature of UnifyingSummary
Schemas Over HTTPStep 1: Setting Up a GitHub RepositoryCreating a GitHub RepositoryUploading Your First SchemaStep 2: Deploying to Cloudflare PagesCreating a New Cloudflare Pages Website ProjectStep 3: Configuring HTTP HeadersInspecting the Current HTTP HeadersDeclaring Custom HTTP Headers on Cloudflare PagesChecking the ResultsStep 4: Creating a Landing PageAdding an HTML Entry PointStep 5: Adding a Custom DomainConfiguring a Custom Domain in Cloudflare PagesSetting Up a CNAME DNS RecordChecking the ResultsBest PracticesSchemas Are ImmutableAdopt a Versioning StrategySummary
First Facet: DataAn Example CSV DatasetA JSON Row RepresentationSecond Facet: StructureGeneral-Purpose ConceptsApplication-Specific ConceptsDataset EntriesThe Dataset SchemaThird Facet: MeaningTimestampIP AddressEmailUS StateCurrencyPriceMilestoneAnalytics EntryFourth Facet: ContextThe Signup Analytics SchemaSummaryAutomated Schema ExtractionNext Steps
Simple Case: Unknown KeywordsExtracting Unknown Keywords as AnnotationsPros and Cons of This ApproachComplex Case: Authoring VocabulariesThe JSON Schema Vocabulary SystemStep 1: Writing a SpecificationStep 2: Writing a Vocabulary Meta-SchemaStep 3: Extending an ImplementationConsuming VocabulariesDefining a DialectMaking Use of the DialectExample: Extracting Annotations with HyperjumpSummary
Introducing the Dataset VocabularyRevisiting the Signup Analytics ExampleJSON Schema BundlingThe Bundling ProcessBundling Our Example Data ProductReferencing Remote DataThe Problem of Streaming JSONIntroducing JSON LinesExtracting MeaningA Simple ExampleUsing Logic OperatorsThe Signup Analytics ExampleDataset LineageFilteringTransformingAggregationSummary
Your Unifying Journey So FarA Constellation of Deeper Principles Guides Unifying1. The Principle of AlignmentTransforming the Abstract to ConcreteWhat You See Can Kill You, and the Same Is True in Data2. The Principle of InformationUnderstanding Uncertainty3. The Principle of LearningDefining LearningDefining Errors4. The Principle of Integrated SimplicityComplexity ReductionDecompositionCompressionMemoizationIntegrating in Communication Networks5. The Principle of ContinuumsMaking Things MeasurableThe Dangers of Misusing MeasurementsA Continuum Example for a Control Strategy Problem6. The Principle of State TransitionsA Simple State MachineSimplifying State Transitions7. The Principle of Decidability What Is Decidability?Two Key Approaches to Problem SolvingMaking Informed DecisionsReal-World Decidability to Reduce Misalignment in Teams8. The Principle of HeuristicsAwareness and Ethical ConsiderationsConnection to Decision Making in Business9. The Principle of MasteryLevels of Mastery in KnowledgeStrategies for Mastery10. The Principle of WisdomSummary
Functional Artificial IntelligenceYour AI Is Only as Good as Your DataBeware Illusions Within Vetting ProcessesQuestion AssumptionsCollective IntelligenceCollaborative IntelligenceUnified IntelligenceCollaborative Learning NetworksPersonalized KnowledgeAnticipatory Design: Personalization and Digital TwinsCodifying Principles of IntelligenceContinuous Human–Machine Learning LoopsApplying Wisdom in PracticeConceptual ZoomabilityWisdom Graphs: Connecting Concepts, Actions, and OutcomesCognitive Primitives: Standardizing Cognitive Experience DesignThe Value of UnifyingPrioritize Knowledge Before AIA Tale of Simple Knowledge Versus Complex IntelligenceFollow the Principle of Integrated SimplicitySummaryGoing Beyond This Book

Content preview from Unifying Business, Data, and Code

Chapter 5. A Universal Language for Data

The limits of my language mean the limits of my world.

Ludwig Wittgenstein

Humans require fluency in a common language in order to communicate complex ideas. In the same way, breaking the data silos of an organization requires every party to use a common language to talk about the structure, semantics, and governance of data. As we saw in Chapter 4, a data definition that describes these concepts in an unambiguous way is called a schema, and it is an essential part of any data-driven strategy.

While there is no shortage of options out there, we choose JSON Schema as our standard data language of choice. In this chapter, you will learn the basics of JSON Schema and how to read and understand any JSON Schema. This chapter purposely takes a modern top-down, vocabulary first approach to teaching you how to think about JSON Schema.

Over the course of this book, you will gain expertise on JSON Schema and be able to use it as the cornerstone of your data-centric innovation initiatives. We will exemplify how schemas can provide immediate value by using them to render basic web forms for our fictitious Intelligence.AI online coffee company.

Note

This chapter builds on Chapter 2 and is aimed at the more technical audience interested in implementing data products. To continue learning about data alignment in a business sense, you can move to Chapter 6.