O'Reilly logo

Mastering Mesos by Akhil Das, Dipa Dubhashi

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Mesos fetcher

The Mesos fetcher is a way by which resources can be downloaded in the task sandbox directory while preparing the task execution. The framework requesting the execution of the task sends a list of CommandInfo::URI values as part of the TaskInfo message, which in turn serves as the Mesos fetcher input.

The Mesos fetcher natively supports the FTP and HTTP protocols, and is also able to copy over files from a filesystem. It also supports all Hadoop client protocols such as Amazon Simple Storage Service (S3), Hadoop distributed Filesystem (HDFS), and so on.

Every Uniform Resource Identifier (URI) that is requested gets downloaded, by default, directly into the sandbox directory. Multiple requests for the same URI results in copies of that ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required