book

Camel in Action, Second Edition

Name: Camel in Action, Second Edition
ISBN: 9781617292934

by Jonathan Anstey, Claus Ibsen

February 2018

Intermediate to advanced

912 pages

28h 11m

English

Manning Publications

Read now

Unlock full access

Cover
Camel in Action
Praise for the First Edition
Copyright
Dedication
Foreword
Foreword
Foreword to the First Edition
Preface
Acknowledgments

About This Book
RoadmapWho should read this bookCode conventionsSource code downloadsSoftware requirementsAuthor Online
About the Authors
About the Cover Illustration
Part 1: First steps
Chapter 1: Meeting Camel
1.1 Introducing Camel1.1.1 What is Camel?1.1.2 Why use Camel?1.2 Getting started1.2.1 Getting Camel1.2.2 Your first Camel ride1.3 Camel’s message model1.3.1 Message1.3.2 Exchange1.4 Camel’s architecture1.4.1 Architecture from 10,000 feet1.4.2 Camel concepts1.5 Your first Camel ride, revisited1.6 Summary
Chapter 2: Routing with Camel
2.1 Introducing Rider Auto Parts2.2 Understanding endpoints2.2.1 Consuming from an FTP endpoint2.2.2 Sending to a JMS endpoint2.3 Creating routes in Java2.3.1 Using RouteBuilder2.3.2 Using the Java DSL2.4 Defining routes in XML2.4.1 Bean injection and Spring2.4.2 The XML DSL2.4.3 Using Camel and Spring2.5 Endpoints revisited2.5.1 Sending to dynamic endpoints2.5.2 Using property placeholders in endpoint URIs2.5.3 Using raw values in endpoint URIs2.5.4 Referencing registry beans in endpoint URIs2.6 Routing and EIPs2.6.1 Using a content-based router2.6.2 Using message filters2.6.3 Using multicasting2.6.4 Using recipient lists2.6.5 Using the wireTap method2.7 Summary and best practices
Part 2: Core Camel
Chapter 3: Transforming data with Camel
3.1 Data transformation overview3.2 Transforming data by using EIPs and Java3.2.1 Using the Message Translator EIP3.2.2 Using the Content Enricher EIP3.3 Transforming XML3.3.1 Transforming XML with XSLT3.3.2 Transforming XML with object marshaling3.4 Transforming with data formats3.4.1 Data formats provided with Camel3.4.2 Using Camel’s CSV data format3.4.3 Using Camel’s Bindy data format3.4.4 Using Camel’s JSON data format3.4.5 Configuring Camel data formats3.5 Transforming with templates3.5.1 Using Apache Velocity3.6 Understanding Camel type converters3.6.1 How the Camel type-converter mechanism works3.6.2 Using Camel type converters3.6.3 Writing your own type converter3.7 Summary and best practices
Chapter 4: Using beans with Camel
4.1 Using beans the hard way and the easy way4.1.1 Invoking a bean from pure Java4.1.2 Invoking a bean defined in XML DSL4.1.3 Using beans the easy way4.2 Understanding the Service Activator pattern4.3 Using Camel’s bean registries4.3.1 JndiRegistry4.3.2 SimpleRegistry4.3.3 ApplicationContextRegistry4.3.4 OsgiServiceRegistry and BlueprintContainerRegistry4.3.5 CdiBeanRegistry4.4 Selecting bean methods4.4.1 How Camel selects bean methods4.4.2 Camel’s method-selection algorithm 4.4.3 Some method-selection examples4.4.4 Potential method-selection problems4.4.5 Method selection using type matching4.5 Performing bean parameter binding4.5.1 Binding with multiple parameters4.5.2 Binding using built-in types4.5.3 Binding using Camel annotations4.5.4 Binding using Camel language annotations4.5.5 Parameter binding using method name with signature4.6 Using beans as predicates and expressions4.6.1 Using beans as predicates in routes4.6.2 Using beans as expressions in routes4.7 Summary and best practices
Chapter 5: Enterprise integration patterns
5.1 Introducing enterprise integration patterns5.1.1 The Aggregator and Splitter EIPs5.1.2 The Routing Slip and Dynamic Router EIPs5.1.3 The Load Balancer EIP5.2 The Aggregator EIP5.2.1 Using the Aggregator EIP5.2.2 Completion conditions for the Aggregator5.2.3 Using persistence with the Aggregator5.2.4 Using recovery with the Aggregator5.3 The Splitter EIP5.3.1 Using the Splitter5.3.2 Using beans for splitting5.3.3 Splitting big messages5.3.4 Aggregating split messages5.3.5 When errors occur during splitting5.4 The Routing Slip EIP5.4.1 Using the Routing Slip EIP5.4.2 Using a bean to compute the routing slip header5.4.3 Using an Expression as the routing slip5.4.4 Using @RoutingSlip annotation5.5 The Dynamic Router EIP5.5.1 Using the Dynamic Router5.5.2 Using the @DynamicRouter annotation5.6 The Load Balancer EIP5.6.1 Introducing the Load Balancer EIP5.6.2 Using load-balancing strategies5.6.3 Using the failover load balancer5.6.4 Using a custom load balancer5.7 Summary and best practices
Chapter 6: Using components
6.1 Overview of Camel components6.1.1 Manually adding components6.1.2 Autodiscovering components6.2 Working with files: File and FTP components6.2.1 Reading and writing files with the File component6.2.2 Accessing remote files with the FTP component6.3 Asynchronous messaging: JMS component6.3.1 Sending and receiving messages6.3.2 Request-reply messaging6.3.3 Message mappings6.4 Networking: Netty4 component6.4.1 Using Netty for network programming6.4.2 Using custom codecs6.5 Working with databases: JDBC and JPA components6.5.1 Accessing data with the JDBC component6.5.2 Persisting objects with the JPA component6.6 In-memory messaging: Direct, Direct-VM, SEDA, and VM components6.6.1 Synchronous messaging with Direct and Direct-VM6.6.2 Asynchronous messaging with SEDA and VM6.7 Automating tasks: Scheduler and Quartz2 components6.7.1 Using the Scheduler component6.7.2 Enterprise scheduling with Quartz6.8 Working with email6.8.1 Sending mail with SMTP6.8.2 Receiving mail with IMAP6.9 Summary and best practices
Part 3: Developing and testing
Chapter 7: Microservices
7.1 Microservices overview7.1.1 Small in size7.1.2 Observable7.1.3 Designed for failure7.1.4 Highly configurable7.1.5 Smart endpoints and dumb pipes7.1.6 Testable7.2 Running Camel microservices7.2.1 Standalone Camel as microservice7.2.2 CDI Camel as microservice7.2.3 WildFly Swarm with Camel as microservice7.2.4 Spring Boot with Camel as microservice7.3 Calling other microservices7.3.1 Recommendation prototype7.3.2 Shopping cart prototype7.3.3 Rules and inventory prototypes7.3.4 Rating prototype7.3.5 Putting all the microservices together7.4 Designing for failures7.4.1 Using the Retry pattern to handle failures7.4.2 Using the Retry pattern without Camel7.4.3 Using Circuit Breaker to handle failures7.4.4 Netflix Hystrix7.4.5 Using Hystrix with Camel7.4.6 Configuring Hystrix7.4.7 Bulkhead pattern 7.4.8 Calling other microservices with fault-tolerance7.4.9 Using Camel Hystrix with Spring Boot7.4.10 The Hystrix dashboard7.5 Summary and best practices
Chapter 8: Developing Camel projects
8.1 Managing projects with Maven8.1.1 Using Camel Maven archetypes8.1.2 Using Maven to add Camel dependencies8.2 Using Camel in Eclipse8.2.1 Creating a new Camel project8.3 Debugging an issue with your new Camel project8.4 Developing custom components8.4.1 Setting up a new Camel component8.4.2 Diving into the implementation8.5 Generating components with the API component framework8.5.1 Generating the skeleton API project8.5.2 Configuring the camel-api-component-maven-plugin8.5.3 Setting advanced configuration options8.5.4 Implementing remaining functionality8.6 Developing data formats8.6.1 Generating the skeleton data format project8.6.2 Writing the custom data format8.7 Summary and best practices
Chapter 9: Testing
9.1 Introducing the Camel Test Kit9.1.1 Using the Camel JUnit extensions9.1.2 Using camel-test to test Java Camel routes9.1.3 Unit testing an existing RouteBuilder class9.2 Testing Camel with Spring, OSGi, and CDI9.2.1 Camel testing with Spring XML9.2.2 Camel testing with Spring Java Config9.2.3 Camel testing with Spring Boot9.2.4 Camel testing with OSGi Blueprint XML9.2.5 Camel testing with CDI9.2.6 Camel testing with WildFly Swarm9.2.7 Camel testing with WildFly9.3 Using the mock component9.3.1 Introducing the mock component9.3.2 Unit-testing with the mock component9.3.3 Verifying that the correct message arrives9.3.4 Using expressions with mocks9.3.5 Using mocks to simulate real components9.4 Simulating errors9.4.1 Simulating errors using a processor9.4.2 Simulating errors using mocks9.4.3 Simulating errors using interceptors9.4.4 Using adviceWith to add interceptors to an existing route9.4.5 Using adviceWith to manipulate routes for testing9.4.6 Using weave with adviceWith to amend routes9.5 Camel integration testing9.5.1 Performing integration testing9.5.2 Using NotifyBuilder9.6 Using third-party testing libraries9.6.1 Using Arquillian to test Camel applications9.6.2 Using Pax Exam to test Camel applications9.6.3 Using other testing libraries9.7 Summary and best practices
Chapter 10: RESTful web services
10.1 RESTful services10.1.1 Understanding the principles of a RESTful API 10.1.2 Using JAX-RS with REST services10.1.3 Using Camel in an existing JAX-RS application10.1.4 Using camel-restlet with REST services10.1.5 Using camel-cxf with REST services10.2 The Camel Rest DSL10.2.1 Exploring a quick example of the Rest DSL10.2.2 Understanding how the Rest DSL works10.2.3 Using supported components for the Rest DSL10.2.4 Configuring Rest DSL10.2.5 Using XML and JSON data formats with Rest DSL10.2.6 Calling RESTful services using Rest DSL10.3 API documentation using Swagger10.3.1 Using Swagger with JAX-RS REST services10.3.2 Using Swagger with Rest DSL10.3.3 Documenting Rest DSL services10.3.4 Documenting input, output, and error codes10.3.5 Configuring API documentation10.3.6 Using CORS and the Swagger web console10.4 Summary and best practices
Part 4: Going further with Camel
Chapter 11: Error handling
11.1 Understanding error handling11.1.1 Recoverable and irrecoverable errors11.1.2 Where Camel’s error handling applies11.2 Using error handlers in Camel11.2.1 Using the default error handler11.2.2 The dead letter channel error handler11.2.3 The transaction error handler11.2.4 The no error handler11.2.5 The logging error handler11.2.6 Features of the error handlers11.3 Using error handlers with redelivery11.3.1 An error-handling use case11.3.2 Using redelivery11.3.3 Using DefaultErrorHandler with redelivery 11.3.4 Error handlers and scopes11.3.5 Reusing context-scoped error handlers11.4 Using exception policies11.4.1 Understanding how onException catches exceptions11.4.2 Understanding how onException works with redelivery11.4.3 Understanding how onException can handle exceptions11.4.4 Custom exception handling11.4.5 New exception while handling exception11.4.6 Ignoring exceptions11.4.7 Implementing an error-handler solution11.4.8 Bridging the consumer with Camel’s error handler 11.5 Working with other error-handling features11.5.1 Using onWhen11.5.2 Using onExceptionOccurred11.5.3 Using onRedeliver11.5.4 Using retryWhile11.6 Summary and best practices
Chapter 12: Transactions and idempotency
12.1 Why use transactions?12.1.1 The Rider Auto Parts partner integration application12.1.2 Setting up the JMS broker and the database12.1.3 The story of the lost message12.2 Transaction basics12.2.1 Understanding Spring’s transaction support12.2.2 Adding transactions12.2.3 Testing transactions12.3 The Transactional Client EIP 12.3.1 Using local transactions12.3.2 Using global transactions12.3.3 Transaction starting from a database resource12.3.4 Transaction redeliveries 12.3.5 Using different transaction propagations12.3.6 Returning a custom response when a transaction fails12.4 Compensating for unsupported transactions12.4.1 Introducing UnitOfWork12.4.2 Using synchronization callbacks12.4.3 Using onCompletion12.5 Idempotency12.5.1 Idempotent Consumer EIP12.5.2 Idempotent repositories12.5.3 Clustered idempotent repository12.6 Summary and best practices
Chapter 13: Parallel processing
13.1 Introducing concurrency13.1.1 Running the example without concurrency13.1.2 Using concurrency13.2 Using thread pools13.2.1 Understanding thread pools in Java13.2.2 Using Camel thread pool profiles13.2.3 Creating custom thread pools13.2.4 Using ExecutorServiceManager 13.3 Parallel processing with EIPs13.3.1 Using concurrency with the Threads EIP13.3.2 Using concurrency with the Multicast EIP13.3.3 Using concurrency with the Wire Tap EIP13.4 Using the asynchronous routing engine13.4.1 Hitting the scalability limit13.4.2 Scalability in Camel13.4.3 Components supporting asynchronous processing13.4.4 Asynchronous API13.4.5 Writing a custom asynchronous component13.4.6 Potential issues when using an asynchronous component13.5 Summary and best practices
Chapter 14: Securing Camel
14.1 Securing your configuration14.1.1 Encrypting configuration14.1.2 Decrypting configuration14.2 Web service security14.2.1 Authentication in web services14.2.2 Authenticating web services using JAAS14.3 Payload security14.3.1 Digital signatures14.3.2 Payload encryption14.4 Transport security14.4.1 Defining global SSL configuration14.5 Route authentication and authorization14.5.1 Configuring Spring Security14.6 Summary and best practices
Part 5 : Running and managing Camel
Chapter 15: Running and deploying Camel
15.1 Starting Camel15.1.1 How Camel starts15.1.2 Camel startup options15.1.3 Ordering routes15.1.4 Disabling autostartup15.2 Starting and stopping routes at runtime15.2.1 Using CamelContext to start and stop routes at runtime15.2.2 Using the Control Bus EIP to start and stop routes at runtime15.2.3 Using RoutePolicy to start and stop routes at runtime15.3 Shutting down Camel15.3.1 Graceful shutdown15.4 Deploying Camel15.4.1 Embedded in a Java application15.4.2 Embedded in a web application15.4.3 Embedded in WildFly15.5 Camel and OSGi15.5.1 Setting up Maven to generate an OSGi bundle15.5.2 Installing and running Apache Karaf15.5.3 Using an OSGi Blueprint-based Camel route15.5.4 Deploying the example15.5.5 Using a managed service factory to spin up route instances15.6 Camel and CDI15.7 Summary and best practices
Chapter 16: Management and monitoring
16.1 Monitoring Camel16.1.1 Checking health at the network level16.1.2 Checking health level at the JVM level16.1.3 Checking health at the application level16.2 Using JMX with Camel16.2.1 Using JConsole to manage Camel16.2.2 Using JConsole to remotely manage Camel16.2.3 Using Jolokia to manage Camel16.3 Tracking application activity16.3.1 Using log files16.3.2 Using core logs16.3.3 Using custom logging16.3.4 Using Tracer16.3.5 Using notifications16.4 Managing Camel applications16.4.1 Managing Camel application lifecycles16.4.2 Using Jolokia and hawtio to manage Camel lifecycles16.4.3 Using Control Bus to manage Camel16.5 The Camel management API16.5.1 Accessing the Camel management API using Java16.5.2 Using Camel management API from within Camel16.5.3 Performance statistics16.5.4 Management-enable custom Camel components16.5.5 Management-enable custom Java beans16.6 Summary and best practices
Part 6: Out in the wild
Chapter 17: Clustering
17.1 Clustered HTTP17.2 Clustered Camel routes17.2.1 Active/passive mode17.2.2 Active/active mode17.2.3 Clustered active/passive mode using Hazelcast17.2.4 Clustered active/passive mode using Consul17.2.5 Clustered active/passive mode using ZooKeeper17.3 Clustered JMS17.3.1 Client-side clustering with JMS and ActiveMQ17.4 Clustered Kafka17.4.1 Kafka consumer offset17.4.2 Crashing a JVM with a running Kafka consumer17.5 Clustering caches17.5.1 Clustered cache using Hazelcast17.5.2 Clustered cache using JCache and Infinispan17.6 Using clustered scheduling17.6.1 Clustered scheduling using Quartz17.7 Calling clustered services using the Service Call EIP17.7.1 How the Service Call EIP works17.7.2 Service Call using static service registry17.7.3 Service Call with failover17.7.4 Configuring Service Call EIP17.7.5 Service Call URI templating17.7.6 Service Call using Spring Boot Cloud and Consul17.8 Summary and best practices
Chapter 18: Microservices with Docker and Kubernetes
18.1 Getting started with Camel on Docker18.1.1 Building and running Camel microservices locally18.1.2 Building and running Camel microservices using Docker18.1.3 Building a Docker image using the Docker Maven plugin18.1.4 Running Java microservices on Docker 18.2 Getting started with Kubernetes18.2.1 Installing Minikube18.2.2 Starting Minikube18.3 Running Camel and other applications in Kubernetes18.3.1 Running applications using kubectl18.3.2 Calling a service running inside a Kubernetes cluster18.3.3 Running Java applications in Kubernetes using Maven tooling18.3.4 Java microservices calling each other in the cluster18.3.5 Debugging Java applications in Kubernetes18.4 Understanding Kubernetes18.4.1 Introducing Kubernetes18.4.2 Kubernetes architecture18.4.3 Essential Kubernetes concepts18.5 Building resilient Camel microservices on Kubernetes18.5.1 Scaling up microservices18.5.2 Using readiness and liveness probes18.5.3 Dealing with failures by calling services in Kubernetes18.6 Testing Camel microservices on Kubernetes18.6.1 Setting up Arquillian Cube18.6.2 Writing a basic unit test using Arquillian Cube18.6.3 Running Arquillian Cube tests on Kubernetes18.6.4 Writing a unit test that calls a Kubernetes service18.7 Introducing fabric8, Helm, and OpenShift18.7.1 fabric818.7.2 Kubernetes Helm18.7.3 OpenShift18.8 Summary and best practices
Chapter 19: Camel tooling
19.1 Camel editors19.1.1 JBoss Fuse Tooling19.1.2 Apache Camel IDEA plugin19.1.3 Camel validation using Maven19.2 Camel Catalog: the information goldmine19.3 hawtio: a web console for Camel and Java applications19.3.1 Understanding hawtio functionality19.3.2 Debugging Camel routes using hawtio19.4 Summary and best practices
Chapter 20: Reactive Camel
20.1 Using Reactive Streams with Camel20.1.1 Reactive Streams API20.1.2 Reactive flow control with back pressure20.1.3 First steps with Reactive Streams20.1.4 Using Camel with Reactive Streams20.1.5 Controlling back pressure from the producer side20.1.6 Controlling back pressure from the consumer side20.2 Using Vert.x with Camel20.2.1 Building a football simulator using Vert.x20.2.2 Using Camel together with Vert.x20.2.3 Summary of using Camel with Vert.x for microservices20.3 Summary and best practices
Chapter 21: Camel and the IoT
21.1 The Internet of Things shopping list21.1.1 Raspberry Pi21.1.2 SD card for Raspberry Pi21.1.3 Power bank for Raspberry Pi21.1.4 Camera for Raspberry Pi21.1.5 TI SensorTag21.2 The Internet of Things architecture21.3 Why Camel is the right choice for the IoT21.3.1 Components21.3.2 Data formats21.3.3 Redelivery21.3.4 Throttling21.3.5 Content-based routing21.3.6 Client-side load balancing21.3.7 Control bus21.4 Gateway-to-data-center connectivity21.4.1 Understanding the architecture21.4.2 Choosing a protocol21.5 Camel and Eclipse Kura21.5.1 Starting Kura in emulator mode21.5.2 Defining Camel routes using the Kura web UI21.5.3 Next steps with Camel and Kura21.6 Next steps with Camel and the IoTLWM2M and the Eclipse Leshan projectEclipse Hono21.7 Summary
Appendix A: Simple, the expression language
A.1 Introducing SimpleA.2 SyntaxA.3 Built-in variablesA.4 Built-in functionsA.5 Built-in file variablesA.6 Built-in operatorsA.6.1 Combining expressionsA.7 The OGNL featureA.8 Using Simple from custom Java codeSummary
Appendix B: The Camel community
Apache Camel websiteJIRA, mailing lists, Gitter, and IRCCamel at GitHubCamel at Stack OverflowCommercial Camel offeringsCamel toolingCamel-extra projectBecoming a Camel committerVideosOther resources
Index

Content preview from Camel in Action, Second Edition

Part 4 Going further with Camel

In complex enterprise systems, lots of things can go wrong. That’s why Camel features an extensive set of error-handling abilities. In chapter 11 we’ll discuss these in detail.

One concept you’ll encounter in enterprise applications is transactions. In chapter 12, we’ll explain how to use Spring’s transaction framework to let Camel routes participate in transactions. In case of a lack of transactional support, you can use compensation and the idempotency patterns, which are covered as well.

In chapter 13 we’ll discuss the important, sometimes complex topic of concurrency. Understanding how to configure and tune threading in Camel is a must for performance-centric projects. We’ll also cover how you can improve ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781617292934Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Camel in Action, Second Edition

by Jonathan Anstey, Claus Ibsen

Part 4 Going further with Camel

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.