O'Reilly logo

Python Social Media Analytics by Michal Krystyanczuk, Siddhartha Chatterjee

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Distributed computing with Celery

Distributed computing is where we use multiple different machines to perform a task. For example, if we wanted to perform some operation on every sentence in a large file, instead of using a single process on a single machine, we could break the file down to many smaller files and assign those files to different machines to process in parallel.

As exciting as this sounds, distributed computing has its own set of problems. Shared memory is not possible as tasks are performed on completely different machines, synchronization between machines becomes a challenge, failure handling becomes more complicated, and so on. To ease our problems, we will use a software that makes the task of distributed computing much ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required