Integrating Django with Scrapy

To make paths easy to call, we remove the external scrapy_spider folder so that inside the movie_reviews_analyzer_app, the webmining_server folder is at the same level as the scrapy_spider folder:

├── db.sqlite3
├── scrapy.cfg
├── scrapy_spider
│   ├── ...
│   ├── spiders
│   │   ...
└── webmining_server

We set the Django path into the Scrapy file:

# Setting up django's project full path.
import sys
sys.path.insert(0, BASE_DIR+'/webmining_server')
# Setting up django's settings module name.
os.environ['DJANGO_SETTINGS_MODULE'] = 'webmining_server.settings'
#import django to load models(otherwise AppRegistryNotReady: Models aren't loaded yet):
import django

Now we can install the library that will ...

Get Machine Learning for the Web now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.